Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
Intro:
I've seen lots of topics here on SO about software testing and other terms I don't understand.
Problem:
As a beginner developer I, unfortunately, have no idea how software testing works, not even how to test a simple function. This is a shame, but thats the truth. I also hope this question can help others beginners developers too.
Question:
Can you help me to understand this subject a little bit more?
Maybe some questions to start would help:
When I develop a function, how should I test it? For example: when working with a sum function, should I test every input value possible or just some limits? How about testing functions with strings as parameters?
In a big program, do I have to test every single piece of code of it? When you guys program do you test every code written?
How automated test works and how can I try one? How tools for automated testing works and what they do?
I`ve heard about unit testing. Can I have a brief explanation on this?
What is a testing framework?
If possible please post some code with examples to clarify the ideas.
Any help on this topic is very welcome! Thanks.
Let's start with the obvious:
How does testing work ? In test-driven development you first think about the functionality you want to implement and then write a test for it. In the example you gave of a sum function it is quite obvious what it should do. You then write a test that makes sure that the summation worked.
Unit tests should be as lightweight as possible so you can run them each time you hit the build button. When you do this now in this example your test will fail, because you have not yet implemented a sum function.
Now you write the actual function and continue debugging and implementing until the test passes. Then you are sure you have implemented the feature you wanted.
Now how should you design your test ? You cannot test all, that is impossible. As an example lets say you take user input you have to validate. Then it would be a natural thing to write at least two test cases for that validation code: one that makes sure that valid input is parsed as such. The second test case gets invalid input and you make sure that it fails, raises an exception or whatever behavior you desired. So in this case it is good to have a positive test which is expected to pass and a negative test which checks if invalid input is not validated.
When should one test? As I mentioned before the test should be lightweight enough so that they can be run on each build. And yes, run all of them. This makes sure that you do not miss a dependency in your code that breaks things far away from the point you edited.
Can anything be tested ? Well, usually methods that rely on external ressources are hard to test. What I mean by that are databases, network connections or specific hardware and drivers. It can be done but then you have to set up a larger test setup.
Bugfixing and tests A typical scenario where test get really useful is if you are squeezing bugs. Not literally of course. If you have a bug you have to fix, first try to write a test for it. And then fix your code until your test passes. From this point on this test "watches over your code" that this bug will never come back again.
What do you gain by testing ? In my opinion there are many things
More modular, easier to maintain code because it has to be testable
Confidence. Having a code base that is largely tested gives you the confidence that it works as expected and stays like this.
You find bugs early. This means you can fix them more easily.
It takes some effort to get used to using tests but I think it is worth it. Especially if you are writing libraries of some kind.
Well,
Usually, there're three kinds of tests. Unit Tests, System tests and QA tests. Unit tests, as the name suggests, test small units - separate functions and classes.
For all modern development environments there're unit test frameworks. There's Nunit for .net, as well as MS unit test framework in Visual Studio, CPPUnit for C++, JUnit, etc. All meant for one thing: connect to parts of your program, run your pre-defined scripts and report error.
CPPUnit, for example, is based on macros like CPPUNIT_ASSERT_EQUAL, meant to be used as something like this: CPPUNIT_ASSERT_EQUAL(sum(arr), 17). And it would say if it's not equal, in which case the test will be considered failed.
You are supposed to develop the tests for every function, and after that - you are not afraid to change and optimize the code. It is generally referred to a "repeatability" - ability to do a complex action, such as full testing of all codebase, with a single click.
Unit tests are required for every moden software development, because the experience shows that they improve development speed. It is also suggested that unit test code may serve as a kind of "documentation" for the library.
System tests are automated tests of larger, high-level functionality. The idea of system tests is to feed clean input (such as databases, user input, etc) to test the whole thing, validating the output against pre-defined resutls. It is essential that the system ouput is deterministic, and depends only on the input. Every time the system changes, the system tests change also.
TDD is a cool-sounding bad idea, suggesting that you should not develop anything before implementing the proper automated tests, and then writing code to satisfy the tests. It is regardedd as failure, because changes in the design are inevitable during development, and a small design change usually causes drastic changes in the unit tests.
Manual QA is the final, and most important type of software testing. The idea is to prepare a test plan, whcih is done during design and coding phases, collecting all ideas developers had during coding of every if statement, how to actually make this particular if statement run along the less expected code path. The QA personnel, meant to be capable of anything that can be done with the program without development environment, can follow the resulting test procedure and their own ideas, to find more bugs.
I found the book 'JUnit Pocket Guide' by Kent Beck as an excellent (and cheap and compact!) introduction to Unit Testing: the book is split roughly into sections on the merits of test-driven programming and general testing techniques and then goes into the specifics of the JUnit framework (which he co-created).
http://www.amazon.co.uk/JUnit-Pocket-Guide-Kent-Beck/dp/0596007434/ref=sr_1_7?ie=UTF8&s=books&qid=1276811600&sr=8-7
With regard to your request for some illustrative examples of Unit Testing; this JUnit Primer isn't bad:
http://www.clarkware.com/articles/JUnitPrimer.html#testcase
What to test
There are a couple of really useful concepts that helped me work this out:
http://en.wikipedia.org/wiki/Equivalence_partitioning
http://en.wikipedia.org/wiki/Boundary-value_analysis
Equivalence partitioning says "Running the test in this context with these events is equivalent to this other test over here, so I don't need to do both."
Boundary analysis says "Here's the point at which a change in context or a different event causes a different outcome, so I will want tests on either side of this boundary."
Thinking about these can really help to minimize the number of tests you need to write.
When to test
I always manually test my code to make sure that it works. I often write an additional unit test which covers every line of code I'm about to write. The unit test fails before I write the code to make it pass. If it's something a user will do, then I often write an acceptance test or automated scenario, too.
Automated testing
Automated testing means either scripting or recording a test so that you don't have to do it manually - your script or code will do it for you. There are two kinds of automation tools: BDD- or English-readable acceptance-testing fixtures which let you write the scripts, and automation wrappers which let you automate more easily. So for instance, you might choose to use GivWenZen or Fitnesse as your fixture, and Selenium as your web automation tool.
If you've got Visual Studio 2008+ you could download the source code from here and try out the scenario to see it run:
http://code.google.com/p/wipflash/
Unit testing
In its strictest sense, Unit Testing means testing an element of code in isolation from all other elements of code. We use mocks in place of real pieces of code that our Code Under Test requires. In reality we are often pragmatic about the nature of "units" - I don't mock out domain objects, for instance.
However, unit testing in the way that most people use it isn't really about testing. Mostly it's a mechanism for achieving clean design and separation of responsibilities, because you're forced into this by trying to test it.
I like to think of unit testing as "writing an example of how I'm going to use the code I'm about to write". The example happens to reside in a unit-testing framework and be executable.
Testing frameworks
Unit-testing frameworks help you to run a lot of tests together.
Acceptance-testing frameworks help you to run a larger scenario and are often English-readable.
Automation frameworks usually sit underneath your English-readable frameworks and help you to interact with an application.
Good luck!
You can divide the tests into three big branches (actually there are more, but if you are a beginner you have to understand the base first): Alpha, Beta, Full working code.
In alpha you try every single value for every single piece of code.
In beta you give the presumably working application to the world, in order to get a feedback from testers and improve it.
When your app is fully working you just have to wait for the error-report (it always comes) of a user who discovered that issue and fix it with a patch.
When I develop a function, how should I test it? For example: when working with a sum function, should I test every input value possible or just some limits? How about testing functions with strings as parameters?
In alpha you have to think about ALL the possibilities your code can be dial with.
In the case of a sum() function, they are a few, but if you are developing let's say a big database server application, you need to take in account every possible input and handle every possible error, think of the end user as a stupid person or even as a malicious cracker.
In a big program, do I have to test every single piece of code of it? When you guys program do you test every code written?
I do, you should, microsoft doesn't. (catch the joke ;) )
How automated test works and how can I try one? How tools for automated testing works and what they do?
No automation can overcome human intelligence and a good debugger (that's the only tool you really need!)
I`ve heard about unit testing. Can I have a brief explanation on this?
http://en.wikipedia.org/wiki/Unit_testing
What is a testing framework?
http://en.wikipedia.org/wiki/Test_automation_framework
When I develop a function, how should I test it? For example: when working with a sum function, should I test every input value possible or just some limits? How about testing functions with strings as parameters? You can use boundary value analysis black box testing type
In a big program, do I have to test every single piece of code of it? When you guys program do you test every code written? Yes, You have to confirm each module working as per expected results
How automated test works and how can I try one? How tools for automated testing works and what they do? Automation Testing depends on different factors ex. Project scope , Time, cost , available resource etc . Recommend you explore automation testing concepts
I`ve heard about unit testing. Can I have a brief explanation on this?
What is a testing framework? . Depends on your programming language selection because in python unittest provided and java have another unit test framework
Overall need to go through Automation testing concepts and application.
When I develop a function, how should I test it? For example: when working with a sum function, should I test every input value possible or just some limits? How about testing functions with strings as parameters?
First it starts with defining the functionality of the application which is used to create the test case. Either you are testing manually or through automation, you need to define output that you want to achieve on every input to identify the limits of test case.
In a big program, do I have to test every single piece of code of it? When you guys program do you test every code written?
Testing can be consuming at times since it is a process that needs to be followed in parts. The code is put to test in parts for every defined functionality. However, if you are working on an application that has a scope for code reusability from the past codes that performs fine, then that part could either be skipped or worked through automation rather than manually checking the code and the output.
How automated test works and how can I try one? How tools for automated testing works and what they do?
Automated tests are usually done through automation test tools that are made to run test codes for checking a software or application. For example, selenium is an open-source test automation tool that can automate tests using web browsers. It provides you interface to write test scripts in various programming languages (Ruby, Java, NodeJS, PHP, Perl, Python, and C#) for different browsers and operating systems.
I`ve heard about unit testing. Can I have a brief explanation on this?
Unit testing is a testing practice in which individual functions of the software are tested. The primary purpose of unit testing is to check every unit or smallest function of application, usually which involves one or a few inputs that leads to common output.
What is a testing framework?
Testing framework is a combination of tools and procedures that are followed for testing an application, software or website. In short, it is a set of rules for designing test cases.
Related
I was reading the Joel Test 2010 and it reminded me of an issue i had with unit testing.
How do i really unit test something? I dont unit test functions? only full classes? What if i have 15 classes that are <20lines. Should i write a 35line unit test for each class bringing 15*20 lines to 15*(20+35) lines (that's from 300 to 825, nearly 3x more code).
If a class is used by only two other classes in the module, should i unit test it or would the test against the other two classes suffice? what if they are all < 30lines of code should i bother?
If i write code to dump data and i never need to read it such as another app is used. The other app isnt command line or it is but no way to verify if the data is good. Do i still need to unit test it?
What if the app is a utility and the total is <500lines of code. Or is used that week and will be used in the future but always need to be reconfiguration because it is meant for a quick batch process and each project will require tweaks because the desire output is unchanged. (i'm trying to say theres no way around it, for valid reasons it will always be tweaked) do i unit test it and if so how? (maybe we dont care if we break a feature used in the past but not in the present or future).
etc.
I think this should be a wiki. Maybe people would like to say an exactly of what they should unit test (or should not)? maybe links to books are good. I tried one but it never clarified what should be unit tested, just the problems of writing unit testing and solutions.
Also if classes are meant to only be in that project (by design, spec or whatever other reason) and the class isnt useful alone (lets say it generates the html using data that returns html ready comments) do i really need to test it? say by checking if all public functions allow null comment objects when my project doesnt ever use null comment. Its those kind of things that make me wonder if i am unit testing the wrong code. Also tons of classes are throwaway when the project. Its the borderline throwaway or not very useful alone code which bothers me.
Here's what I'm hearing, whether you meant it this way or not: a whole litany of issues and excuses why unit testing might not be applicable to your code. In other words: "I don't see what I'll be getting out of unit tests, and they're a lot of bother to write; maybe they're not for me?"
You know what? You may be right. Unit tests are not a panacea. There are huge, wide swaths of testing that unit testing can't cover.
I think, though, that you're misestimating the cost of maintenance, and what things can break in your code. So here are my thoughts:
Should I test small classes? Yes, if there are things in that class that can possibly break.
Should I test functions? Yes, if there are things in this function that can possibly break. Why wouldn't you? Or is your concern over whether it's considered a unit or not? That's just quibbling over names, and shouldn't have any bearing on whether you should write unit tests for it! But it's common in my experience to see a method or function described as a unit under test.
Should I unit test a class if it's used by two other classes? Yes, if there's anything that can possibly break in that class. Should I test it separately? The advantage of doing so is to be able to isolate breakages straight down to the shared class, instead of hunting through the using classes to see if it was they that broke or one of their dependencies.
Should I test data output from my class if another program will read it? Hell yes, especially if that other program is a 3rd-party one! This is a great application of unit tests (or perhaps system tests, depending on the isolation involved in the test): to prove to yourself that the data you output is precisely what you think you should have output. I think you'll find that has the power to simplify support calls immeasurably. (Though please note it's not a substitute for good acceptance testing on that customer's end.)
Should I test throwaway code? Possibly. Will pursuing a TDD strategy get your throwaway code out the door faster? It might. Will having solid unit-tested chunks that you can adapt to new constraints reduce the need to throw code away? Perhaps.
Should I test code that's constantly changing? Yes. Just make sure all applicable tests are brought up to date and pass! Constantly changing code can be particularly susceptible to errors, after all, and enabling safe change is another of unit testing's great benefits. Plus, it probably puts a burden on your invariant code to be as robust as possible, to enable this velocity of change. And you know how you can convince yourself whether a piece of code is robust...
Should I test features that are no longer needed? No, you can remove the test, and probably the code as well (testing to ensure you didn't break anything in the process, of course!). Don't leave unit test rot around, especially if the test no longer works or runs, or people in your org will move away from unit tests and you'll lose the benefit. I've seen this happen. It's not pretty.
Should I test code that doesn't get used by my project, even if it was written in the context of my project? Depends on what the deliverable of your project is, and what the priorities of your project are. But are you sure nobody outside of your project will use it? If they won't, and you aren't, perhaps it's just dead code, in which case see above. From my point of view, I wouldn't feel I'd done a complete job with a class if my testing didn't cover all its important functionality, whether the project used all that functionality or not. I like classes that feel complete, but I keep an eye towards not overengineering a bunch of stuff I don't need. If I put something in a class, then, I intend for it to be used, and will therefore want to make sure it works. It's an issue of personal quality and satisfaction to me.
Don't get fixated on counting lines of code. Write as much test code as you need to convince yourself that every key piece of functionality is being thoroughly tested. As an extreme example, the SQLite project has a tests:source-code ratio of more than 600:1. I use the term "extreme" in a good sense here; the ludicrous amount of testing that goes on is possibly the predominant reason that SQLite has taken over the world.
How can you do all those calculations? Ideally you should never be in a situation where you could count the lines of your completed class and then start writting the unit test from scratch. Those 2 types of code (real code and test code) should be developed and evolved together, and the only LOC metric that should really worry you in the end is 0 LOCs for test code.
Relative LOC counts for code and tests are pointless. What matters more is test coverage. What matters most is finding the bugs.
When I'm writing unit tests, I tend to focus my efforts on testing complicated code that is more likely to contain bugs. Simple stuff (e.g. simple getter and setter methods) is unlikely to contain bugs, and can be tested indirectly by higher-level unit tests.
Some Time ago, i had The same question you have posted in mind. I studied a lot of articles, Tutorials, books and so on... Although These resources give me a good starting point, i still was insecure about how To apply efficiently Unit Testing code. After coming across xUnit Test Patterns: Refactoring Test Code and put it in my shelf for about one year (You know, we have a lot of stuffs To study), it gives me what i need To apply efficiently Unit Testing code. With a lot of useful patterns (and advices), you will see how you can become an Unit Testing coder. Topics as
Test strategy patterns
Basic patterns
Fixture setup patterns
Result verification patterns
Test double patterns
Test organization patterns
Database patterns
Value patterns
And so on...
I will show you, for instance, derived value pattern
A derived input is often employed when we need to test a method that takes a complex object as an argument. For example, thorough input validation testing requires we exercise the method with each of the attributes of the object set to one or more possible invalid values. Because The first rejected value could cause Termination of The method, we must verify each bad attribute in a separate call. We can instantiate The invalid object easily by first creating a valid object and then replacing one of its attributes with a invalid value.
A Test organization pattern which is related To your question (Testcase class per feature)
As The number of Test methods grows, we need To decide on which Testcase class To put each Test method... Using a Testcase class per feature gives us a systematic way To break up a large Testcase class into several smaller ones without having To change out Test methods.
But before reading
(source: xunitpatterns.com)
My advice: read carefully
You seem to be concerned that there could be more test-code than the code-under-test.
I think the ratios could we be higher than you say. I would expect any serious test to exercise a wide range of inputs. So your 20 line class might well have 200 lines of test code.
I do not see that as a problem. The interesting thing for me is that writing tests doesn't seem to slow me down. Rather it makes me focus on the code as I write it.
So, yes test everything. Try not to think of testing as a chore.
I am part of a team that have just started adding test code to our existing, and rather old, code base.
I use 'test' here because I feel that it can be very vague as to weather it is a unit test, or a system test, or an integration test, or whatever. The differences between the terms have large grey areas, and don't add a lot of value.
Because we live in the real world, we don't have time to add test code for all of the existing functionality. We still have Dave the test guy, who finds most bugs. Instead, as we develop we write tests. You know how you run your code before you tell your boss that it works? Well, use a unit framework (we use Junit) to do those runs. And just keep them all, rather than deleting them. Whatever you normally do to convince yourself that it works. Do that.
If it is easy to write the code, do it. If not, leave it to Dave until you think of a good way to do automate it, or until you get that spare time between projects where 'they' are trying to decide what to put into the next release.
for java u can use junit
JUnit
JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks.
* Getting Started
* Documentation
* JUnit related sites/projects
* Mailing Lists
* Get Involved
Getting Started
To get started with unit testing and JUnit read the article: JUnit Cookbook.
This article describes basic test writing using JUnit 4.
You find additional samples in the org.junit.samples package:
* SimpleTest.java - some simple test cases
* VectorTest.java - test cases for java.util.Vector
JUnit 4.x only comes with a textual TestRunner. For graphical feedback, most major IDE's support JUnit 4. If necessary, you can run JUnit 4 tests in a JUnit 3 environment by adding the following method to each test class:
public static Test suite() {
return new JUnit4TestAdapter(ThisClass.class);
}
Documentation
JUnit Cookbook
A cookbook for implementing tests with JUnit.
Javadoc
API documentation generated with javadoc.
Frequently asked questions
Some frequently asked questions about using JUnit.
Release notes
Latest JUnit release notes
License
The terms of the common public license used for JUnit.
The following documents still describe JUnit 3.8.
The JUnit 3.8 version of this homepage
Test Infected - Programmers Love Writing Tests
An article demonstrating the development process with JUnit.
JUnit - A cooks tour
JUnit Related Projects/Sites
* junit.org - a site for software developers using JUnit. It provides instructions for how to integrate JUnit with development tools like JBuilder and VisualAge/Java. As well as articles about and extensions to JUnit.
* XProgramming.com - various implementations of the xUnit testing framework architecture.
Mailing Lists
There are three junit mailing lists:
* JUnit announce: junit-announce#lists.sourceforge.net Archives/Subscribe/Unsubscribe
* JUnit users list: junit#yahoogroups.com Archives/Subscribe/Unsubscribe
* JUnit developer list: junit-devel#lists.sourceforge.net Archives/Subscribe/Unsubscribe
Get Involved
JUnit celebrates programmers testing their own software. As a result bugs, patches, and feature requests which include JUnit TestCases have a better chance of being addressed than those without.
JUnit source code is now hosted on GitHub.
One possibility is to reduce the 'test code' to a language that describes your tests, and an interpreter to run the tests. Teams I have been a part of have used this to wonderful ends, allowing us to write significantly more tests than the "lines of code" would have indicated.
This allowed our tests to be written much more quickly and greatly increased the test legibility.
I am going to answer what I believe are the main points of your question. First, how much test-code should you write? Well, Test-Driven Development can be of some help here. I do not use it as strictly as it is proposed in theory, but I find that writing a test first often helps me to understand the problem I want to solve much better. Also, it will usually lead to good test-coverage.
Secondly, which classes should you test? Again, TDD (or more precisely some of the principles behind it) can be of help. If you develop your system top down and write your tests first, you will have tests for the outer class when writing the inner class. These tests should fail if the inner class has bugs.
TDD is also tightly coupled with the idea of Design for Testability.
My answer is not intended to solve all your problems, but to give you some ideas.
I think it's impossible to write a comprehensive guide of exactly what you should and shouldn't unit test. There are simply too many permutations and types of objects, classes, and functions, to be able to cover them all.
I suggest applying personal responsibility to the testing, and determining the answer yourself. It's your code, and you're responsible for it working. If it breaks, you have to pay the consequences of fixing the code, repairing the data, taking responsibility for the lost revenue, and apologizing to the people whose application broke while they were trying to use it. Bottom line - your code should never break. So what do you have to do to ensure this?
Sometimes unit testing can work well to help you test out all of the specific methods in a library. Sometimes unit testing is just busy-work, because you can tell the code is working based on your use of the code during higher-level testing. You're the developer, you're responsible for making sure the code never breaks - what do you think is the best way to achieve that?
If you think unit testing is a waste of time in a specific circumstance - it probably is. If you've tested the code in all of the application use-case scenarios and they all work, the code is probably good.
If anything is happening in the code that you don't understand - even if the end result is acceptable - then you need to do some more testing to make sure there's nothing you don't understand.
To me, this seems like common sense.
Unit testing is mostly for testing your units from aspect of functionality. You can test and see if a specific input come, will we receive the expected value or will we throw the right exception?
Unit tests are very useful. I recommend you to write down these tests. However, not everything is required to be tested. For example, you don't need to test simple getters and setters.
If you want to write your unit tests in Java via Eclipse, please look at "How To Write Java Unit Tests". I hope it helps.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
Could anyone suggest books or materials to learn unit test?
Some people consider codes without unit tests as legacy codes. Nowadays, Test Driven Development is the approach for managing big software projects with ease. I like C++ a lot, I learnt it on my own without any formal education. I never looked into Unit Test before, so feel left out. I think Unit Tests are important and would be helpful in the long run. I would appreciate any help on this topic.
My main points of concern are:
What is a Unit Test? Is it a comprehensive list of test cases which should be analyzed? So lets us a say i have a class called "Complex Numbers" with some methods in it (lets says finding conjugate, an overloaded assignment operator and an overloaded multiplication operator. What should be typical test cases for such a class? Is there any methodology for selecting test cases?
Are there any frameworks which can create unit tests for me or i have to write my own class for tests? I see an option of "Test" in Visual Studio 2008, but never got it working.
What is the criteria for Units tests? Should there be a unit test for each and every function in a class? Does it make sense to have Unit Tests for each and every class?
An important point (that I didn't realise in the beginning) is that Unit Testing is a testing technique that can be used by itself, without the need to apply the full Test Driven methodology.
For example, you have a legacy application that you want to improve by adding unit tests to problem areas, or you want to find bugs in an existing app. Now you write a unit test to expose the problem code and then fix it. These are semi test-driven, but can completely fit in with your current (non-TDD) development process.
Two books I've found useful are:
Test Driven Development in Microsoft .NET
A very hands on look at Test Driven development, following on from Kent Becks' original TDD book.
Pragmatic Unit Testing with C# and nUnit
It comes straight to the point what unit testing is, and how to apply it.
In response to your points:
A Unit test, in practical terms is a single method in a class that contains just enough code to test one aspect / behaviour of your application. Therefore you will often have many very simple unit tests, each testing a small part of your application code. In nUnit for example, you create a TestFixture class that contains any number of test methods. The key point is that the tests "test a unit" of your code, ie a smallest (sensible) unit as possible. You don't test the underlying API's you use, just the code you have written.
There are frameworks that can take some of the grunt work out of creating test classes, however I don't recommmend them. To create useful unit tests that actually provide a safety net for refactoring, there is no alternative but for a developer to put thought into what and how to test their code. If you start becoming dependent on generating unit tests, it is all too easy to see them as just another task that has to be done. If you find yourself in this situation you're doing it completely wrong.
There are no simple rules as to how many unit tests per class, per method etc. You need to look at your application code and make an educated assessment of where the complexity exists and write more tests for these areas. Most people start by testing public methods only because these in turn usually exercise the remainder of the private methods. However this is not always the case and sometimes it is necessary to test private methods.
In short, even experienced unit testers start by writing obvious unit tests, then look for more subtle tests that become clearer once they have written the obvious tests. They don't expect to get every test up-front, but instead add them as they come to their mind.
While you've already accepted an answer to your question I'd like to recommend a few other books not yet mentioned:
Working Effectively with Legacy Code - Michael Feathers - As far as I know this is the only book to adequately tackle the topic of turning existing code that wasn't designed for testability into testable code. Written as more of a reference manual, its broken down into three sections: An overview of the tools and techniques, A series of topical guides to common road blocks in legacy code, A set of specific dependency breaking techniques referenced throughout the rest of the book.
Agile Principles, Patterns, and Practices - Robert C. Martin - Examples in java, there is a sequel with examples in C#. Both are easy to adapt to C++
Clean Code:A Handbook of Agile Software Craftsmanship - Robert C. Martin - Martin describes this as a prequel to his APPP books and I would agree. This book makes a case for professionalism and self-discipline, two essential qualities in any serious software developer.
The two books by Robert (Uncle Bob) Martin cover much more material than just Unit testing but they drive home just how beneficial unit testing can be to code quality and productivity. I find myself referring to these three books on a regular basis.
In .NET I strongly recommend "The Art of Unit Testing" by Roy Osherove, it is very comprehensive and full of good advice.
Nowadays, Test Driven Development is
the approach for managing big software
projects with ease.
That is because TDD allows you to make sure after each change that everything that worked before the change still works, and if it doesn't it allows you to pinpoint what was broken, much easier. (see at the end)
What is a Unit Test? Is it a
comprehensive list of test cases which
should be analyzed?
A Unit Test is a piece of code that asks a "unit" of your code to perform an operation, then verifies that the operation was indeed performed and the result is as expected. If the result is not correct, it raises / logs an error.
So lets us a say i have a class called
"Complex Numbers" with some methods in
it (lets says finding conjugate, an
overloaded assignment operator and an
overloaded multiplication operator.
What should be typical test cases for
such a class? Is there any methodology
for selecting test cases?
Ideally, you would test all the code.
when you create an instance of the
class, it is created with the correct
default values
when you ask it to find the
conjugates, it does finds the correct
ones (also test border cases, like the
conjugate for zero)
when you assign a value the value is
assigned and displayed correctly
when you multiply a complex by a
value, it is multiplied correctly
Are their any frameworks which can
create unit tests for me or i have to
write my own class for tests?
See CppUnit
I see an option of "Test" in Visual
Studio 2008, but never got it working.
Not sure on that. I haven't used VS 2008 but it may be available just for .NET.
What is the criteria for Units tests?
Should there be a unit test for each
and every function in a class? Does it
make sense to have Unit Tests for each
and every class?
Yes, it does. While that is an awful lot of code to write (and maintain with every change) the price is worth paying for large projects: It guarantees that your changes to the code base do what you want them to and nothing else.
Also, when you make a change, you need to update the unit-tests for that change (so that they pass again).
In TDD, you first decide what you want the code to do (say, your complex numbers class), then write the tests that verify those operations, then write the class so that the tests compile and execute correctly (and nothing more).
This ensures that you write the minimal code possible (and don't over-complicate the design of the complex class) and it also ensures that your code does what it does. At the end of writing the code, you have a way to test it's functionality and ensuring it's correctness.
You also have an example of using the code that you will be able to access at any point.
For further reading/documentation, look into "dependency injection" and method stubs as used in unit testing and TDD.
With test driven design, you normally want to write the tests first. They should cover the operations you're actually using/going to use. I.e. unless they're necessary for the client code to do its job, they shouldn't exist. Selecting test cases is something of an art. There are obvious things like testing boundary conditions, but in the end, nobody's found a really reliable, systematic way of assuring that tests (unit or otherwise) cover all the conditions that matter.
Yes, there are frameworks. A couple of the best known are:
Boost Unit Test Framework
CPPUNit
CPPUnit is a port of JUnit, so those who've used JUnit previously will probably find it comfortable. Otherwise, I'd tend to recommend Boost -- they also have a Test Library to help write the individual tests -- rather a handy addition.
Unit tests should be sufficient to ensure that the code works. If (for example) you have a private function that's used internally, you generally don't need to test it directly. Instead, you test whatever provides the public interface. As long as that works correctly, it's no business of the outside world how it does its job. Of course, in some cases it's easier to test little pieces, and when it is, that's perfectly legitimate -- but ultimately you care about the visible interface, not the internals. Certainly the whole external interface should be exercised, and test cases generally chosen to exercise the paths through the code. Again, there's nothing massively different about unit tests versus other kinds. It's mostly just a more systematic way of applying normal testing techniques.
Unit tests are simply a way to exercise a given body of code to ensure that a defined set of conditions leads to the expected set of out comes. As Steven points out, these "exercises" should check across a range of criteria ("BICEP"). Yes, ideally you should test all of your classes and all of the methods in these classes although there is always some room for judgement: testing shouldn't be an end in itself but rather should support the wider project goals.
Ok, so...theory is nice but to really understand Unit Testing, my recommendation would be to pull together the appropriate tools and just get started. Like most things in programming, if you have the right tools, it is easy to learn by doing.
First, pick up a copy of NUnit. It is free, easy to install and easy to work with. If you'd like some documentation, check out Pragmatic Unit Testing in C# with NUnit.
Next, go to http://www.testdriven.net/ and get a copy of TestDriven.net. It installs into Visual Studio 2008 and gives you right-click access to a full range of testing tools including the ability to run NUnit tests against a file, directory or project (typically, tests are written in a separate project). You can also run tests with debugging or, coolest of all, run all the tests against a copy of NCover. NCover will show you exactly what code is being exercised so you can figure out where you need to improve your test coverage. TestDriven.net costs $170 for a professional license but, if you are like me, it will very quickly become an integral tool in your toolbox. Anyway, I've found it to be an excellent professional investment.
Good luck!
I can't answer your question for Visual Studio 2008, but I know that Netbeans has a few integrated tools for you to use.
The code coverage two allows for you to see which paths have been checked, and how much of the code is actually covered by the unit tests.
It has the support for the unit tests built in.
As far as quality of tests I'm borrowing a bit from the "Pragmatic Unit Testing in Java with JUnit" by Andrew Hunt and David Thomas:
Unit testing should check for BICEP:
Boundary, Inverse relationships, Cross-checking, Error conditions, and Performance.
Also quality of the tests are determined by A-TRIP:
Automatic, Thorough, Repeatable, Independent, and Professional.
Here's something on when not to write unit tests (i.e. on when it's viable and even preferable to skip unit testing): Should one test internal implementation, or only test public behaviour?
The short answer is:
When you can automate integration tests (because it's important to have automated tests, but those tests don't have to be unit tests)
When it's cheap to run the integration test suite (no good if it takes two days to run, or if you can't afford to let every developer have access to an integration test equipment)
When it isn't necessary to find bugs before integration testing (which depends in part on whether components are developed separately or incrementally)
Buy the book "xUnit Test Patterns: Refactoring Test Code". Its very excellent. It does cover high-level strategy decisions as well as low level test patterns.
Nowadays, Test Driven Development is the approach for managing big software projects with ease.
TDD built on unit tests but they are different. You don't need to be use TDD to make use of unit tests. My personal preference is to write test first, but I don't feel I do the whole TDD thing.
What is a Unit Test?
A Unit Test is a bit of code that tests the behaviour of one unit. How one unit is defined differs between people. But in general they are:
Quick to run
Independent from each other
Test only a small part (a unit ;) of your code base.
Binary outcome - That is it passes or fails.
Should only test one outcome of the unit (for each outcome create a different unit test)
Repeatable
Are their any frameworks which can create unit tests
To write the tests - Yes but I've never seen anyone say anything nice about them.
To help you write & run tests, a whole bunch of them.
Should there be a unit test for each and every function in a class?
You have a few different camps in this - the 100%ers would say yes. Every method must be tested and you should have 100% code coverage. The other extreme is that unit tests should only cover areas that you have even encounter bugs or you expect to find bugs. The middle ground (and the stand I take) is to unit tests everything that is not "too simple to break". Setters/getters and anything that just calls a single other method. I aim to have 80% code coverage and a low CRAP factor (so a low chance I've been naughty and decided to not test something as it was "too complex to test).
The book that helped me "get" unit tests JUnit in Action. Sorry I don't do much in the C++ world, so I can not suggest a C++ based alternative.
I primarily spend my time working on automated tests of win32 and .NET applications, which take about 30% of our time to write and 70% to maintain. We have been looking into methods of reducing the maintenance times, and have already moved to a reusable test library that covers most of the key components of our software. In addition we have some work in progress to get our library to a state where we can use keyword based testing.
I have been considering unit testing our test library, but I'm wondering if it would be worth the time. I'm a strong proponent of unit testing of software, but I'm not sure how to handle test code.
Do you think automated Gui testing libraries should be unit tested? Or is it just a waste of time?
First of all I've found it very useful to look at unit-test as "executable specifications" instead of tests. I write down what I want my code to do and then implement it. Most of the benefits I get from writing unit tests is that they drive the implementation process and focus my thinking. The fact that they're reusable to test my code is almost a happy coincidence.
Testing tests seems just a way to move the problem instead of solving it. Who is going to test the tests that test the tests? The 'trick' that TDD uses to make sure tests are actually useful is by making them fail first. This might be something you can use here too. Write the test, see it fail, then fix the code.
I dont think you should unit test your unit tests.
But, if you have written your own testing library, with custom assertions, keyboard controllers, button testers or what ever, then yes. You should write unit tests to verify that they all work as intented.
The NUnit library is unit tested for example.
In theory, it is software and thus should be unit-tested. If you are rolling your own Unit Testing library, especially, you'll want to unit test it as you go.
However, the actual unit tests for your primary software system should never grow large enough to need unit testing. If they are so complex that they need unit testing, you need some serious refactoring of your software and some attention to simplifying your unit tests.
You might want to take a look at Who tests the tests.
The short answer is that the code tests the tests, and the tests test the code.
Huh?
Testing Atomic Clocks
Let me start with an analogy. Suppose you are
travelling with an atomic clock. How would you know that the clock is
calibrated correctly?
One way is to ask your neighbor with an atomic clock (because everyone
carries one around) and compare the two. If they both report the same
time, then you have a high degree of confidence they are both correct.
If they are different, then you know one or the other is wrong.
So in this situation, if the only question you are asking is, "Is my
clock giving the correct time?", then do you really need a third clock
to test the second clock and a fourth clock to test the third? Not if
all. Stack Overflow avoided!
IMPO: it's a tradeoff between how much time you have and how much quality you'd like to have.
If I would be using a home made test harnas, I'd test it if time permits.
If it's a third party tool I'm using, I'd expect the supplier to have tested it.
There really isn't a reason why you could/shouldn't unit test your library. Some parts might be too hard to unit test properly, but most of it probably can be unit tested with no particular problem.
It's actually probably particularly beneficial to unit test this kind of code, since you expect it to be both reliable and reusable.
The tests test the code, and the code tests the tests. When you say the same intention in two different ways (once in tests and once in code), the probability of both of them being wrong is very low (unless already the requirements were wrong). This can be compared to the dual entry bookkeeping used by accountants. See http://butunclebob.com/ArticleS.UncleBob.TheSensitivityProblem
Recently there has been discussion about this same issue in the comments of http://blog.objectmentor.com/articles/2009/01/31/quality-doesnt-matter-that-much-jeff-and-joel
About your question, that should GUI testing libraries be tested... If I understood right, you are making your own testing library, and you want to know if you should test your testing library. Yes. To be able to rely on the library to report tests correctly, you should have tests which make sure that library does not report any false positives or false negatives. Regardless of whether the tests are unit tests, integration tests or acceptance tests, there should be at least some tests.
Usually writing unit tests after the code has been written is too late, because then the code tends to be more coupled. The unit tests force the code to be more decoupled, because otherwise small units (a class or a closely related group of classes) can not be tested in isolation.
When the code has already been written, then usually you can add only integration tests and acceptance tests. They will be run with the whole system running, so you can make sure that the features work right, but covering every corner case and execution path is harder than with unit tests.
We generally use these rules of thumb:
1) All product code has both unit tests (arranged to correspond closely with product code classes and functions), and separate functional tests (arranged by user-visible features)
2) Do not write tests for 3rd party code, such as .NET controls, or third party libraries. The exception to this is if you know they contain a bug which you are working around. A regression test for this (which fails when the 3rd party bug disappears) will alert you when upgrades to your 3rd party libraries fix the bug, meaning you can then remove your workarounds.
3) Unit tests and functional tests are not, themselves, ever directly tested - APART from using the TDD procedure of writing the test before the product code, then running the test to watch it fail. If you don't do this, you will be amazed by how easy it is to accidentally write tests which always pass. Ideally, you would then implement your product code one step at a time, and run the tests after each change, in order to see every single assertion in your test fail, then get implemented and start passing. Then you will see the next assertion fail. In this way, your tests DO get tested, but only while the product code is being written.
4) If we factor out code from our unit or functional tests - creating a testing library which is used in many tests, then we do unit test all of this.
This has served us very well. We seem to have always stuck to these rules 100%, and we are very happy with our arrangement.
Kent Beck's book "Test-Driven Development: By Example" has an example of test-driven development of a unit test framework, so it's certainly possible to test your tests.
I haven't worked with GUIs or .NET, but what concerns do you have about your unit tests?
Are you worried that it may describe the target code as incorrect when it is functioning properly? I suppose this is a possibility, but you'd probably be able to detect that if this was happening.
Or are you concerned that it may describe the target code as functioning properly even if it isn't? If you're worried about that, then mutation testing may be what you're after. Mutation testing changes parts of code being tested, to see if those changes cause any tests to fail. If it doesn't, then either the code isn't being run, or the results of that code isn't being tested.
If mutation testing software isn't available on your system, then you could do the mutation manually, by sabotaging the target code yourself and seeing if it causes the unit tests to fail.
If you're building a suite of unit testing products that aren't tied to a particular application, then maybe you should build a trivial application that you can run your test software on and ensure it gets the failures and successes expected.
One problem with mutation testing is that it doesn't ensure that the tests cover all potential scenarios a program may encounter. Instead, it only ensures that the scenarios anticipated by the target code are all tested.
Answer
Yes, your GUI testing libraries should be tested.
For example, if your library provides a Check method to verify the contents of a grid against a 2-dimensional array, you want to be sure that it works as intended.
Otherwise, your more complex test cases that test business processes in which a grid must receive particular data may be unreliable. If an error in your Check method produces false negatives, you'll quickly find the problem. However, if it produces false positives, you're in for major headaches down the line.
To test your CheckGrid method:
Populate a grid with known values
Call the CheckGrid method with the values populated
If this case passes, at least one aspect of CheckGrid works.
For the second case, you're expecting the CheckGrid method to report a test failure.
The particulars of how you indicate the expectation will depend on your xUnit framework (see an example later). But basically, if the Test Failure is not reported by CheckGrid, then the test case itself must fail.
Finally, you may want a few more test case for special conditions, such as: empty grids, grid size mismatching array size.
You should be able to modify the following dunit example for most frameworks in order to test that CheckGrid correctly detects errors:
begin
//Populate TheGrid
try
CheckGrid(<incorrect values>, TheGrid);
LFlagTestFailure := False;
except
on E: ETestFailure do
LFlagTestFailure := True;
end;
Check(LFlagTestFailure, 'CheckGrid method did not detect errors in grid content');
end;
Let me reiterate: your GUI testing libraries should be tested; and the trick is - how do you do so effectively?
The TDD process recommends that you first figure out how you intend testing a new piece of functionality before you actually implement it. The reason is, that if you don't, you often find yourself scratching your head as to how you're going to verify it works. It is extremely difficult to retrofit test cases onto existing implementations.
Side Note
One thing you said bothers me a little... you said it takes "70% (of your time) to maintain (your tests)"
This sounds a little wrong to me, because ideally your tests should be simple, and should themselves only need to change if your interfaces or rules change.
I may have misunderstood you, but I got the impression that you don't write "production" code. Otherwise you should have more control over the cycle of switching between test code and production code so as to reduce your problem.
Some suggestions:
Watch out for non-deterministic values. For example, dates and artificial keys can play havoc with certain tests. You need a clear strategy of how you'll resolve this. (Another answer on its own.)
You'll need to work closely with the "production developers" to ensure that aspects of the interfaces you're testing can stabilise. I.e. They need to be cognisant of how your tests identify and interact with GUI components so they don't arbitrarily break your tests with changes that "don't affect them".
On the previous point, it would help if automated tests are run whenever they make changes.
You should also be wary of too many tests that simply boil down to arbitrary permutations. For example, if each customer has a category A, B, C, or D; then 4 "New Customer" tests (1 for each category) gives you 3 extra tests that don't really tell you much more than the first one, and are 'hard' to maintain.
Personally, I don't unit test my automation libraries, I run them against a modified version of the baseline to ensure all the checkpoints work. The principal here is that my automation is primarily for regression testing, e.g. that the results for the current run are the same as the expect results (typically this equates to the results of the last run). By running the tests against a suitably modified set of expected results, all the tests shoud fail. If they don't you have a bug in your test suite. This is a concept borrowed from mutation testing that I find works well for checking GUI automation suites.
From your question, I can understand that you are building a Keyword Driven Framework for performing automation testing. In this case, it is always recommended to do some white box testing on the common and GUI utility functions. Since you are interested in Unit testing each GUI testing functionality in your libraries, please go for it. Testing is always good. It is not a waste of time, I would see it as a 'value-add' to your framework.
You have also mentioned about handling test code, if you mean the test approach, please group up different functions/modules performing similar work eg: GUI element validation (presence), GUI element input, GUI element read. Group for different element types and perform a type unit test approach for each group. It would be easier for you to track the testing. Cheers!
I would suggest test the test is a good idea and something that must be done. Just make sure that what you're building to test your app is not more complex that the app itself. As it was said before, TDD is a good approach even when building automated functional tests (I personally wouldn't do it like that, but it is a good approach anyway). Unit testing you test code is a good approach as well. IMHO, if you're automating GUI testing, just go ahead with whatever manual tests are available (you should have steps, raw scenarios, expected results and so on), make sure they pass. Then, for other test that you might create and that are not already manually scripted, unit test them and follow a TDD approach. (then if you have time you could unit test the other ones).
Finally, keyword driven, is, IMO, the best approach you could follow because it gives you the most flexible approach.
You may want to explore a mutation testing framework ( if you work with Java : check out PIT Mutation Testing ). Another way to assess the quality of your unit testing is to look at reports provided by tools such as SonarQube ; the reports include various coverage metrics;
I'm looking to unit testing as a means of regression testing on a project.
However, my issue is that the project is basically a glorified DIR command -- it performs regular expression tests and MD5 filters on the results, and allows many criteria to be specified, but the entire thing is designed to process input from the system on which it runs.
I'm also a one-man development team, and I question the value of a test for code written by me which is written by me.
Is unit testing worthwhile in this situation? If so, how might such tests be accomplished?
EDIT: MD5 and Regex functions aren't provided by me -- they are provided by the Crypto++ library and Boost, respectively. Therefore I don't gain much by testing them. Most of the code I have simply feeds data into the libraries, and the prints out the results.
The value of test-after, the way you are asking, can indeed be limited in certain circumstances, but the way to unit test, from the description would be to isolate the regular expression tests and MD5 filters into one section of code, and abstract the feeding of the input so that in production it feeds from the system, and during the unit test, your test class passes in that input.
You then collect a sampling of the different scenarios you intend to support, and feed those in via different unit tests that exercise each scenario.
I think the value of the unit test will come through if you have to change the code to handle new scenarios. You will be confident that the old scenarios don't break as you make changes.
Is unit testing worthwhile in this situation?
Not necessarily: especially for a one-man team I think it may be sufficient to have automated testing of something larger than a "unit" ... further details at "Should one test internal implementation, or only test public behaviour?"
Unit testing can still provide value in a one-man show. It gives you confidence in the functionality and correctness (at some level) of the module. But some design considerations may be needed to help make testing more applicable to your code. Modularization makes a big difference, especially if combined with some kind of dependency injection, instead of tight coupling. This allows test versions of collaborators to be used for testing a module in isolation. In your case, a mock file system object could return a predictable set of data, so your filtering and criteria code can be evaluated.
The value of regression testing is often not realized until it's automated. Once that's done, things become a lot easier.
That means you have to be able to start from a known position (if you're generating MD5s on files, you have to start with the same files each time). Then get one successful run where you can save the output - that's the baseline.
From that point on, regression testing is simply a push-button job. Start your test, collect the output and compare it to your known baseline (of course, if the output ever changes, you'll need to check it manually, or with another independent script before saving it as the new baseline).
Keep in mind the idea of regression testing is to catch any bugs introduced by new code (i.e., regressing the software). It's not to test the functionality of that new code.
The more you can automate this, the better, even as a one-man development team.
When you were writing the code, did you test it as you went? Those tests could be written into an automated script, so that when you have to make a change in 2 months time, you can re-run all those tests to ensure you haven't regressed something.
In my experience the chance of regression increases sharply depending on how much time goes by after the time you finish version 1 and start coding version 2, because you'll typically forget the subtle nuances of how it works under different conditions - your unit tests are a way of encoding those nuances.
An integration test against the filesystem would be worthwhile. Just make sure it does what it needs to do.
Is unit testing valuable in a one-man shop scenario? Absolutely! There's nothing better than being able to refactor your code with absolute confidence you haven't broken anything.
How do I unit test this? Mock the system calls, and then test the rest of your logic.
I question the value of a test for code written by me which is written by me
Well, that's true now but in a year it will be you, the one-year-more-experienced developer developing against software written by you-now, the less experienced and knowledgeable developer (by comparison). Won't you want the code written by that less experienced guy (you a year ago) to be properly tested so you can make changes with confidence that nothing has broken?
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I saw many questions asking 'how' to unit test in a specific language, but no question asking 'what', 'why', and 'when'.
What is it?
What does it do for me?
Why should I use it?
When should I use it (also when not)?
What are some common pitfalls and misconceptions
Unit testing is, roughly speaking, testing bits of your code in isolation with test code. The immediate advantages that come to mind are:
Running the tests becomes automate-able and repeatable
You can test at a much more granular level than point-and-click testing via a GUI
Note that if your test code writes to a file, opens a database connection or does something over the network, it's more appropriately categorized as an integration test. Integration tests are a good thing, but should not be confused with unit tests. Unit test code should be short, sweet and quick to execute.
Another way to look at unit testing is that you write the tests first. This is known as Test-Driven Development (TDD for short). TDD brings additional advantages:
You don't write speculative "I might need this in the future" code -- just enough to make the tests pass
The code you've written is always covered by tests
By writing the test first, you're forced into thinking about how you want to call the code, which usually improves the design of the code in the long run.
If you're not doing unit testing now, I recommend you get started on it. Get a good book, practically any xUnit-book will do because the concepts are very much transferable between them.
Sometimes writing unit tests can be painful. When it gets that way, try to find someone to help you, and resist the temptation to "just write the damn code". Unit testing is a lot like washing the dishes. It's not always pleasant, but it keeps your metaphorical kitchen clean, and you really want it to be clean. :)
Edit: One misconception comes to mind, although I'm not sure if it's so common. I've heard a project manager say that unit tests made the team write all the code twice. If it looks and feels that way, well, you're doing it wrong. Not only does writing the tests usually speed up development, but it also gives you a convenient "now I'm done" indicator that you wouldn't have otherwise.
I don't disagree with Dan (although a better choice may just be not to answer)...but...
Unit testing is the process of writing code to test the behavior and functionality of your system.
Obviously tests improve the quality of your code, but that's just a superficial benefit of unit testing. The real benefits are to:
Make it easier to change the technical implementation while making sure you don't change the behavior (refactoring). Properly unit tested code can be aggressively refactored/cleaned up with little chance of breaking anything without noticing it.
Give developers confidence when adding behavior or making fixes.
Document your code
Indicate areas of your code that are tightly coupled. It's hard to unit test code that's tightly coupled
Provide a means to use your API and look for difficulties early on
Indicates methods and classes that aren't very cohesive
You should unit test because its in your interest to deliver a maintainable and quality product to your client.
I'd suggest you use it for any system, or part of a system, which models real-world behavior. In other words, it's particularly well suited for enterprise development. I would not use it for throw-away/utility programs. I would not use it for parts of a system that are problematic to test (UI is a common example, but that isn't always the case)
The greatest pitfall is that developers test too large a unit, or they consider a method a unit. This is particularly true if you don't understand Inversion of Control - in which case your unit tests will always turn into end-to-end integration testing. Unit test should test individual behaviors - and most methods have many behaviors.
The greatest misconception is that programmers shouldn't test. Only bad or lazy programmers believe that. Should the guy building your roof not test it? Should the doctor replacing a heart valve not test the new valve? Only a programmer can test that his code does what he intended it to do (QA can test edge cases - how code behaves when it's told to do things the programmer didn't intend, and the client can do acceptance test - does the code do what what the client paid for it to do)
The main difference of unit testing, as opposed to "just opening a new project and test this specific code" is that it's automated, thus repeatable.
If you test your code manually, it may convince you that the code is working perfectly - in its current state. But what about a week later, when you made a slight modification in it? Are you willing to retest it again by hand whenever anything changes in your code? Most probably not :-(
But if you can run your tests anytime, with a single click, exactly the same way, within a few seconds, then they will show you immediately whenever something is broken. And if you also integrate the unit tests into your automated build process, they will alert you to bugs even in cases where a seemingly completely unrelated change broke something in a distant part of the codebase - when it would not even occur to you that there is a need to retest that particular functionality.
This is the main advantage of unit tests over hand testing. But wait, there is more:
unit tests shorten the development feedback loop dramatically: with a separate testing department it may take weeks for you to know that there is a bug in your code, by which time you have already forgotten much of the context, thus it may take you hours to find and fix the bug; OTOH with unit tests, the feedback cycle is measured in seconds, and the bug fix process is typically along the lines of an "oh sh*t, I forgot to check for that condition here" :-)
unit tests effectively document (your understanding of) the behaviour of your code
unit testing forces you to reevaluate your design choices, which results in simpler, cleaner design
Unit testing frameworks, in turn, make it easy for you to write and run your tests.
I was never taught unit testing at university, and it took me a while to "get" it. I read about it, went "ah, right, automated testing, that could be cool I guess", and then I forgot about it.
It took quite a bit longer before I really figured out the point: Let's say you're working on a large system and you write a small module. It compiles, you put it through its paces, it works great, you move on to the next task. Nine months down the line and two versions later someone else makes a change to some seemingly unrelated part of the program, and it breaks the module. Worse, they test their changes, and their code works, but they don't test your module; hell, they may not even know your module exists.
And now you've got a problem: broken code is in the trunk and nobody even knows. The best case is an internal tester finds it before you ship, but fixing code that late in the game is expensive. And if no internal tester finds it...well, that can get very expensive indeed.
The solution is unit tests. They'll catch problems when you write code - which is fine - but you could have done that by hand. The real payoff is that they'll catch problems nine months down the line when you're now working on a completely different project, but a summer intern thinks it'll look tidier if those parameters were in alphabetical order - and then the unit test you wrote way back fails, and someone throws things at the intern until he changes the parameter order back. That's the "why" of unit tests. :-)
Chipping in on the philosophical pros of unit testing and TDD here are a few of they key "lightbulb" observations which struck me on my tentative first steps on the road to TDD enlightenment (none original or necessarily news)...
TDD does NOT mean writing twice the amount of code. Test code is typically fairly quick and painless to write and is a key part of your design process and critically.
TDD helps you to realize when to stop coding! Your tests give you confidence that you've done enough for now and can stop tweaking and move on to the next thing.
The tests and the code work together to achieve better code. Your code could be bad / buggy. Your TEST could be bad / buggy. In TDD you are banking on the chances of BOTH being bad / buggy being fairly low. Often its the test that needs fixing but that's still a good outcome.
TDD helps with coding constipation. You know that feeling that you have so much to do you barely know where to start? It's Friday afternoon, if you just procrastinate for a couple more hours... TDD allows you to flesh out very quickly what you think you need to do, and gets your coding moving quickly. Also, like lab rats, I think we all respond to that big green light and work harder to see it again!
In a similar vein, these designer types can SEE what they're working on. They can wander off for a juice / cigarette / iphone break and return to a monitor that immediately gives them a visual cue as to where they got to. TDD gives us something similar. It's easier to see where we got to when life intervenes...
I think it was Fowler who said: "Imperfect tests, run frequently, are much better than perfect tests that are never written at all". I interprete this as giving me permission to write tests where I think they'll be most useful even if the rest of my code coverage is woefully incomplete.
TDD helps in all kinds of surprising ways down the line. Good unit tests can help document what something is supposed to do, they can help you migrate code from one project to another and give you an unwarranted feeling of superiority over your non-testing colleagues :)
This presentation is an excellent introduction to all the yummy goodness testing entails.
I would like to recommend the xUnit Testing Patterns book by Gerard Meszaros. It's large but is a great resource on unit testing. Here is a link to his web site where he discusses the basics of unit testing. http://xunitpatterns.com/XUnitBasics.html
I use unit tests to save time.
When building business logic (or data access) testing functionality can often involve typing stuff into a lot of screens that may or may not be finished yet. Automating these tests saves time.
For me unit tests are a kind of modularised test harness. There is usually at least one test per public function. I write additional tests to cover various behaviours.
All the special cases that you thought of when developing the code can be recorded in the code in the unit tests. The unit tests also become a source of examples on how to use the code.
It is a lot faster for me to discover that my new code breaks something in my unit tests then to check in the code and have some front-end developer find a problem.
For data access testing I try to write tests that either have no change or clean up after themselves.
Unit tests aren’t going to be able to solve all the testing requirements. They will be able to save development time and test core parts of the application.
This is my take on it. I would say unit testing is the practice of writing software tests to verify that your real software does what it is meant to. This started with jUnit in the Java world and has become a best practice in PHP as well with SimpleTest and phpUnit. It's a core practice of Extreme Programming and helps you to be sure that your software still works as intended after editing. If you have sufficient test coverage, you can do major refactoring, bug fixing or add features rapidly with much less fear of introducing other problems.
It's most effective when all unit tests can be run automatically.
Unit testing is generally associated with OO development. The basic idea is to create a script which sets up the environment for your code and then exercises it; you write assertions, specify the intended output that you should receive and then execute your test script using a framework such as those mentioned above.
The framework will run all the tests against your code and then report back success or failure of each test. phpUnit is run from the Linux command line by default, though there are HTTP interfaces available for it. SimpleTest is web-based by nature and is much easier to get up and running, IMO. In combination with xDebug, phpUnit can give you automated statistics for code coverage which some people find very useful.
Some teams write hooks from their subversion repository so that unit tests are run automatically whenever you commit changes.
It's good practice to keep your unit tests in the same repository as your application.
LibrarIES like NUnit, xUnit or JUnit are just mandatory if you want to develop your projects using the TDD approach popularized by Kent Beck:
You can read Introduction to Test Driven Development (TDD) or Kent Beck's book Test Driven Development: By Example.
Then, if you want to be sure your tests cover a "good" part of your code, you can use software like NCover, JCover, PartCover or whatever. They'll tell you the coverage percentage of your code. Depending on how much you're adept at TDD, you'll know if you've practiced it well enough :)
Unit-testing is the testing of a unit of code (e.g. a single function) without the need for the infrastructure that that unit of code relies on. i.e. test it in isolation.
If, for example, the function that you're testing connects to a database and does an update, in a unit test you might not want to do that update. You would if it were an integration test but in this case it's not.
So a unit test would exercise the functionality enclosed in the "function" you're testing without side effects of the database update.
Say your function retrieved some numbers from a database and then performed a standard deviation calculation. What are you trying to test here? That the standard deviation is calculated correctly or that the data is returned from the database?
In a unit test you just want to test that the standard deviation is calculated correctly. In an integration test you want to test the standard deviation calculation and the database retrieval.
Unit testing is about writing code that tests your application code.
The Unit part of the name is about the intention to test small units of code (one method for example) at a time.
xUnit is there to help with this testing - they are frameworks that assist with this. Part of that is automated test runners that tell you what test fail and which ones pass.
They also have facilities to setup common code that you need in each test before hand and tear it down when all tests have finished.
You can have a test to check that an expected exception has been thrown, without having to write the whole try catch block yourself.
I think the point that you don't understand is that unit testing frameworks like NUnit (and the like) will help you in automating small to medium-sized tests. Usually you can run the tests in a GUI (that's the case with NUnit, for instance) by simply clicking a button and then - hopefully - see the progress bar stay green. If it turns red, the framework shows you which test failed and what exactly went wrong. In a normal unit test, you often use assertions, e.g. Assert.AreEqual(expectedValue, actualValue, "some description") - so if the two values are unequal you will see an error saying "some description: expected <expectedValue> but was <actualValue>".
So as a conclusion unit testing will make testing faster and a lot more comfortable for developers. You can run all the unit tests before committing new code so that you don't break the build process of other developers on the same project.
Use Testivus. All you need to know is right there :)
Unit testing is a practice to make sure that the function or module which you are going to implement is going to behave as expected (requirements) and also to make sure how it behaves in scenarios like boundary conditions, and invalid input.
xUnit, NUnit, mbUnit, etc. are tools which help you in writing the tests.
Test Driven Development has sort of taken over the term Unit Test. As an old timer I will mention the more generic definition of it.
Unit Test also means testing a single component in a larger system. This single component could be a dll, exe, class library, etc. It could even be a single system in a multi-system application. So ultimately Unit Test ends up being the testing of whatever you want to call a single piece of a larger system.
You would then move up to integrated or system testing by testing how all the components work together.
First of all, whether speaking about Unit testing or any other kinds of automated testing (Integration, Load, UI testing etc.), the key difference from what you suggest is that it is automated, repeatable and it doesn't require any human resources to be consumed (= nobody has to perform the tests, they usually run at a press of a button).
I went to a presentation on unit testing at FoxForward 2007 and was told never to unit test anything that works with data. After all, if you test on live data, the results are unpredictable, and if you don't test on live data, you're not actually testing the code you wrote. Unfortunately, that's most of the coding I do these days. :-)
I did take a shot at TDD recently when I was writing a routine to save and restore settings. First, I verified that I could create the storage object. Then, that it had the method I needed to call. Then, that I could call it. Then, that I could pass it parameters. Then, that I could pass it specific parameters. And so on, until I was finally verifying that it would save the specified setting, allow me to change it, and then restore it, for several different syntaxes.
I didn't get to the end, because I needed-the-routine-now-dammit, but it was a good exercise.
What do you do if you are given a pile of crap and seem like you are stuck in a perpetual state of cleanup that you know with the addition of any new feature or code can break the current set because the current software is like a house of cards?
How can we do unit testing then?
You start small. The project I just got into had no unit testing until a few months ago. When coverage was that low, we would simply pick a file that had no coverage and click "add tests".
Right now we're up to over 40%, and we've managed to pick off most of the low-hanging fruit.
(The best part is that even at this low level of coverage, we've already run into many instances of the code doing the wrong thing, and the testing caught it. That's a huge motivator to push people to add more testing.)
This answers why you should be doing unit testing.
The 3 videos below cover unit testing in javascript but the general principles apply across most languages.
Unit Testing: Minutes Now Will Save Hours Later - Eric Mann - https://www.youtube.com/watch?v=_UmmaPe8Bzc
JS Unit Testing (very good) - https://www.youtube.com/watch?v=-IYqgx8JxlU
Writing Testable JavaScript - https://www.youtube.com/watch?v=OzjogCFO4Zo
Now I'm just learning about the subject so I may not be 100% correct and there's more to it than what I'm describing here but my basic understanding of unit testing is that you write some test code (which is kept separate from your main code) that calls a function in your main code with input (arguments) that the function requires and the code then checks if it gets back a valid return value. If it does get back a valid value the unit testing framework that you're using to run the tests shows a green light (all good) if the value is invalid you get a red light and you then can fix the problem straight away before you release the new code to production, without testing you may actually not have caught the error.
So you write tests for you current code and create the code so that it passes the test. Months later you or someone else need to modify the function in your main code, because earlier you had already written test code for that function you now run again and the test may fail because the coder introduced a logic error in the function or return something completely different than what that function is supposed to return. Again without the test in place that error might be hard to track down as it can possibly affect other code as well and will go unnoticed.
Also the fact that you have a computer program that runs through your code and tests it instead of you manually doing it in the browser page by page saves time (unit testing for javascript). Let's say that you modify a function that is used by some script on a web page and it works all well and good for its new intended purpose. But, let's also say for arguments sake that there is another function you have somewhere else in your code that depends on that newly modified function for it to operate properly. This dependent function may now stop working because of the changes that you've made to the first function, however without tests in place that are run automatically by your computer you will not notice that there's a problem with that function until it is actually executed and you'll have to manually navigate to a web page that includes the script which executes the dependent function, only then you notice that there's a bug because of the change that you made to the first function.
To reiterate, having tests that are run while developing your application will catch these kinds of problems as you're coding. Not having the tests in place you'd have to manually go through your whole application and even then it can be hard to spot the bug, naively you send it out into production and after a while a kind user sends you a bug report (which won't be as good as your error messages in a testing framework).
It's quite confusing when you first hear of the subject and you think to yourself, am I not already testing my code? And the code that you've written is working like it is supposed to already, "why do I need another framework?"... Yes you are already testing your code but a computer is better at doing it. You just have to write good enough tests for a function/unit of code once and the rest is taken care of for you by the mighty cpu instead of you having to manually check that all of your code is still working when you make a change to your code.
Also, you don't have to unit test your code if you don't want to but it pays off as your project/code base starts to grow larger as the chances of introducing bugs increases.
Unit-testing and TDD in general enables you to have shorter feedback cycles about the software you are writing. Instead of having a large test phase at the very end of the implementation, you incrementally test everything you write. This increases code quality very much, as you immediately see, where you might have bugs.