Related
I have the C++ code of a exe which contains a UI and some process.
My goal is to separate the UI from the process and to convert the exe into a dll.
In order to do that, I am thinking of generating unit test before touching any code and then to do my modification and make sure the tests are not failing.
The problem is that I am not sure if this is the best approach and if it is, is there a way to automatically generate unit test.
BTW, I am using VS 2012.
Do you have any guidance for me?
It's relatively hard to write meaningful unit tests for GUIs. There are frameworks like FrogLogic's Squish that make GUI testing relatively easy, but most often, these tools are not free.
Note also that it is no small feat to write unit tests "after the fact", as the original code might already have to be changed to make it testable.
As far as I know, there are no tools for automatically bringing existing code under unit tests - if it were that easy, there should be no new bugs at all, right? As arne says in his answer, if code was not designed to be tested in the first place, it usually has to be changed to be testable.
The best you can do in my opinion is to learn some techniques of how to introduce unit tests with relatively few changes (so that you can introduce the unit tests before you start the "real" modifications); one book on this subject I've read recently is Michael Feathers' "Working Effectively with Legacy Code" (Amazon Link: http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052). Although it has some shortcomings, it has pretty detailed descriptions of techniques how you can easily introduce unit tests.
I have been given a source code. I have to implement TDD approach to it.
How ever the general approach for TDD is:
Write the test.
Fail it.
Write the code.
Pass the test.
Refactor, if necessary.
How should I add tests to such an existing source code?
Suggestions for an ad-hoc approach for such a case, is welcome.
Thanks
Don't forget step 0: understand the requirements. What you can do is to discover the requirements, then write the test that demonstrates whether the requirement is satisfied. If it will pass, then great. If it doesn't, you've found a bug. Either way, you've added a regression test.
What you can't do is implement TDD (or any development practice) for code that's already been written: that boat has sailed. What you can do is enable future development on the codebase to benefit from test-driven development practices.
You could treat that code base as third party code and write some learning tests for it. Learning tests will allow you to discover the code while building yourself a nice test suite for further developments should you have to modify the code later.
If the legacy code is small enough, you could do that until code coverage gets near a comfortable 90 or 100%.
I read a lot of posts that convinced me I should start writing unit test, I also started to use dependency injection (Unity) for the sake of easier mocking, but I'm still not quite sure on what stage I should start writing the unit tests and mocks, and how or where to start.
Would the preferred way be writing the unit tests before the methods as described in the TDD methodology?
Is there any different methodology or manner for unit testing?
Test first / test after:
It should be noted that 'test first' as part of TDD is just as much (if not more) to do with design as it is to do with unit testing. It's a software development technique in its own right -- writing the tests results in a constant refining of the design.
On a separate note: If there is one significant advantage to TDD from a pure unit testing perspective, it is that it is much harder (though not impossible) to write a test that's wrong when doing TDD. If you write the test beforehand, it should always fail because the logic required to make the test pass does not yet exist. If you write the test afterwards, the logic should be there, but if the test is bugged or is testing the wrong thing, it may pass regardless.
I.e. if you write a bad test before, you may get a green light when you expect a red (so you know the test is bad). If you write a bad test afterwards, you will get a green light when you expected a green (unaware of the bad test).
Books
The pragmatic unit testing book is well worth a look, as is Roy Osherove's "The Art of Unit Testing". The pragmatic book is more narrowly focussed on the different types of test inputs you can try to find bugs, whereas TAOUT covers a wider spread of topics such as test doubles, strategies, maintainability etc. Either book is good; it depends what you want from it.
Also, here's a link to a talk Roy Osherove did on unit testing. It's worth a watch (so are some of the test review videos he recorded, as he points out various problems and dos/don'ts along with reasons why).
How to start
There's nothing better than writing code. Find a fairly simple class that doesn't reference much else. Then, start writing some tests.
Always ask yourself "what do I want to try and prove with this test?" before you write it, then give it a decent name (usually involving the method being called, the scenario and the expected result, e.g. on a stack: "Pop WhenStackIsEmpty ThrowsException").
Think of all the inputs you can throw at it, different combinations of methods that may yield interesting results and so forth.
If you are curious about unit testing the best way to learn it is try it. You will probably start writing integration tests at first, but that is fine. When they seem too difficult to maintain or too much work to write, read more about what kind of tests there are (unit, functional, integration) and try to learn the difference.
If you are interested in starting with TDD, Uncle Bob is a good source. Particalulary this.
More on unit testing
Ensure that you get consistent test results. Running the same test repeatedly should return the same results consistently.
The tests should not require configuration.
Testing order should not matter. This means partial test runs can work correctly. Also, if you keep this design philosophy in mind, it will likely aid in your test creation.
Remember that any testing is better than no testing. The difficulty can be found in writing good clean unit tests that promote ease of creation and maintenance. The more difficult the testing framework, the more opposition there will be to employing it. Testing is your friend.
In C# and with visual studio I find following procedure very helpful:
Think! Make a small upfront design. You need to have a clear picture what classes you need and how objects should relate with each other. Concentrate only on one class/object (the class/object you want to implement) and one relationship. Otherwise you end up with a too heavyweight design. I often end up with multiple sketches (only a few boxes and arrows) on a spare sheet of paper.
Create the class in the production code and name it appropriately.
Pick one behaviour of the class you want to implement and create a method stub for it. With visual studio creating empty method stubs is a piece of cake.
Write a test for it. Therefor you will need to initialize that object, call the method and make an assert to verify the result. In the easiest case the assertion requires another method stub or a property in the production code.
Compile and let the test runner show you the red bar!
Code the required behavior in the production code to see the green bar appear.
Move to the next behaviour.
For this procedure two things are very important:
You need a clear picture what you want and how the class/object should look like. At least spend some time one it.
Think about behaviours of the class/object. This will make the tests and the production code behaviour-centric, which is a very natural approach to think about classes/objects.
Test first or not test first?
I find it usually harder to retrofitting tests to existing production code. In most cases this is due to tangled dependencies to other objects, when the object which is under test needs to be initialized. TDD usually avoids this, because you want to initialize the object in the test cases with less effort as possible. This will result to a very loose coupling.
When I retrofit tests, the must cumbersome job is the task of initializing the object under test. Also the assertions may be a lot of work because of tangled dependencies. For this you will need to change the production code and break dependencies. By using dependency injection properly this should not be an issue.
Yes, the preferred way of doing TDD is to write the test first (as implied by the name Test-Driven Development). When you start out with TDD it can be hard to know where to start writing the tests, so I would suggest to be pragmatic about it. After all, the most important part of our job is about delivering working code, not so much how the code was crafted.
So you can start by writing tests for existing code. Once you get a hang of how the unit tests are structured, which ones that seem to do a good job and which ones that seem not so god, then you will find it easier to dive more into the test-first approach. I have found that I write tests first to a greater extent as time goes by. It simply becomes more natural with increased experience.
Steve Sanderson has a great writeup on TDD best practices.
http://feeds.codeville.net/~r/SteveCodeville/~3/DWmOM3O0M2s/
Also, there's a great set of tutorials for doing an ASP.net mvc project that discusses a lot TDD principles (if you don't mind learning ASP.net MVC along the way)
http://www.asp.net/learn/mvc-videos/ Look for the "Storefront" series at the bottom of the page.
MOQ seems to be the hot mocking framework lately, you may want to look into that as well
In summary, try to write a test to validate something you'r trying to archive, then implement the code to make it work.
The pattern is known as Red - Green - Refactor. Also do your best to minimize dependencies so that your tests can focus on one component.
Personally, I use Visual Studio Unit Tests. I'm not a hardcore TDD developer, but what i like to do is this:
Create a new project and define a few of the fundamental classes based on the system design (that way I can at least get some intellisense)
create a unit tests project and start writing unit tests to satisfy the functionality i'm trying to implement.
Make them fail
Make them pass (implement)
Refactor
Repeat, try to make the test more stringent or create more tests until i feel its solid.
I also feel its very useful to add functionality onto an exiting code base. If you want to add some new feature, first create the unit test for what you want to add, step through the code to see what you have to change, then go through the TDD process.
Choose a small non-critical application and implement it using TDD. At first the new way of thinking will feel weird, but maybe after a week or two practice it fill feel natural.
Here is a tutorial application (in branch "tutorial") that shows what kinds of tests to write. In that tutorial you write code to pass the predefined test cases, so that you get into the rhythm, and later you then write your own tests. The README file contains instructions. It's written in Java, but you can easily adapt it to C#.
I would take on TDD, test-first development, before mocks and dependency injection. To be sure, mocks can help you better isolate your units - and thus do better unit testing - but to my mind, mocking and DI are more advanced concepts that can interfere with the clarity of just writing your tests first.
Mocks, and DI, have their place; they're good tools to have in your toolbox. But they take some sophistication, a more advanced understanding, than the typical testing neophyte has. Writing your tests first, however, is exactly as simple as it sounds. So it's easier to take on, and it's powerful all by itself (without mocks and DI). You'll get earlier, easier wins by writing mock-free tests first, than by trying to begin with mocks, and TDD, and DI all at once.
Start with test-first; when you are very comfortable with it, and when your code is telling you you need mocks, then take on mocks.
I have worked for companies which take unit testing/integration testing too far and those that do too little so I like to think I have a good balance between the two.
I would recommend TDD - Test Driven Development. This ensures you have good coverage but it also keeps focusing your attention on the right place and problem.
So the first thing you do for every piece of new development is write a unit test - even if you don't have a single class to test.
Think about what you are testing. Now run the test. Why wouldn't it compile? Because you need classA. Create the class and run the test. Why doesn't it compile? Because it doesn't have methodA. Write method one and run unit test again. Why does the test fail? Because methodA isn't implemented. Implement methodA and run test. Why does it fail? Because methodA doesn't return the correct expected value...etc
You continue like this writing unit tests as you develop and then eventually the test will pass and the piece of functionality will be complete.
Extending on Steve Freeman's answer: Dave Astel's book is called "Test-driven Development - A practical guide". If the kind of application you're writing is a GUI application then this should be helpful. I read Kent Becks' book but I couldn't figure out how to start a project with TDD. Astel's book test-drives a complete non-trivial GUI application from start to finish using stories. It helped me a lot to acutally start with TDD, it showed me where and how to start.
Test driven development can be confusing for beginners, a lot of books that I read when I was learning TDD would teach you how to write Unit Tests for a Calculator class but there seems to be very less help for building real world apps, that are more data centric if I dare say. For me the breakthrough was when I understood what is Behaviour Driven Development or BDD and how to start doing testing from outside in. Now I can simply advice you to focus on your application behaviour and write unit tests to verify it. There is a lot of debate going on between TDD and BDD but I think that well written automated tests at every level add value and to write them we need to focus on behaviour.
Hadi Hariri has an excellent post here
http://hadihariri.com/2012/04/11/what-bdd-has-taught-me/
I have also written some articles on the topic that I feel will help in understanding all the concepts related to TDD here
http://codecooked.com/introduction-to-unit-testing-and-test-driven-development/
http://codecooked.com/different-types-of-tests-and-which-ones-to-use/
Read Pragmatic Unit Testing in C# with NUnit. It has comprehensive information about starting writing testes and structuring the code to make it more unit testing friendly.
If you haven't written unit tests before, then just pick some classes and begin to write your unit tests, and continue to work on developing more unit tests.
As you gain experience you can then begin to mock out the database for example, by using the Unity framework, but, I would suggest starting simply and gaining experience before making this leap.
Once you are comfortable with how to write unit tests, then you can try doing TDD.
I prefer KentBeck's approach which is nicely explained in the book, Test Driven Development by Example - Kent Beck.
from your question i can infer you are not sure with the test frame work - choosing the correct test frame work is very important for TDD or writing unit tests(in general).
Only practical problem with TDD is "Refactoring"(we need to refactor test code as well) takes lot of time.
I think Dave Astels' book is still one of the best introductions. It's for Java, but you should be able to translate.
I'm soon to do a 10min Grok talk on Unit Testing at my company. I've been trying it myself, and feel that it can certainly bring benefits to the company. We already do WebInject testing in our dedicated QA team, But I want to try and sell unit testing to the devs.
So with only 10mins what would you cover and why?
we're a Microsoft Shop C# Web Apps, I've used NUnit in my experience.
Unit testing is all about confidence.
It allows you to be confident that your code is solid, and that other people can rely on it when they're writing their own parts of a system. If you can get across that unit testing will help to eliminate the trepidation that comes with the first release of a new system, I would hope that your audience will soon become very interested.
I'd start with a problem a lot of programmers might be familiar with: that is the fear of making a change to existing code because of the fear they might break something. How that prevents work from happening, or prevents it being done properly (because they're afraid to refactor) and so leads to having to rewrite everything every x years.
Unit Testing -> Refactoring -> Living Code.
Edit:
btw, I would Not lead with the 'all code without unit tests is legacy code' quote from Michael Feathers. It certainly made me feel defensive the first time I heard it. By the time people stop feeling affronted the 10 minutes will be over :-) (personally I think that quote is more true than it is helpful).
Here's a good format for a short talk on a technique X:
why you decided to try it X the first place
what you personally have gained from using X
what limitations you've noticed, things that X doesn't address
Don't "sell" or spend lots of time on the theory. Do prepare beforehand and point people to books, URLs of articles or tutorials that you think are most helpful. Those who are interested after your talk can look up the details on the Web.
Try to briefly talk about the aspect of Test Driven Development: write tests first and the interfaces as you go, then implement everything.
Maybe also about continuous integration, this means that as soon as you check something into your source control system, the project gets compiled and all the tests run so the developer knows immediately if he has done something wrong.
If there are any project managers in the audience, also be so fair to tell them that unit testing will make your project take 15-30 % more time, but it will be worth it in the long run.
You could mention that it will be a difficult learning curve, and it will feel like productivity is being impacted, but the benefits are worth it:
e.g. effectively the creation of an automated regression test suite, which in turn allows you to make bigger additions or modifications to existing without worrying that you are breaking some existing functionality.
Creation of production code will be slower, but this should be offset by the higher quality of the code, i.e. fewer bugs, which in the long run means overall higher productivity.
I think 10 minutes is enough to present a simple example of how unit testing can save you time.
Implement a class (you can TDD if you feel like it) and show how a unit test can catch a modification that breaks the class.
Also, you can highlight how you can be faster developing components if you test the isolatedly (i.e. you don't need to bring up your web application, log in, go to your functionality, test); you can just run your tests.
You might be able to perform this on a piece of code from your company- and maybe show how a unit test might have caught a bug you have had recently.
If you give a demonstration, do it on a working piece of code from a project that everyone is familiar with. Avoid contrived examples. Books on TDD are already full of them, and they don't really sell how TDD can work for a real project.
For the love of god, emphasize that unit tests are for testing "units" of logic. I hate looking at a QA suite of NUnit tests that nobody expected to have to maintain, where each "unit test" tests valid outputs for 150 (binary) input files and then shits itself if one fails, without telling you which one.
I would demonstrate:
The confidence it gives in code you produce
The confidence it gives when you change code because it passes the unit tests
The benefits of code coverage, no more "Oh that else statement was never tested!"
The benefits of running unit tests per each build on a CI platform like Hudson
FWIW we run the crappy visual studio testing via MSTEST on our Hudson box and I've got an xslt that Hudson uses to convert the results to the nunit format so Hudson can decipher them. Just putting that out there in case they want you to stick with a Microsoft testing platform.
Accountability, as highlighted by Kent Beck, is another trait that unit testing facilitates. Listen to his podcast at IT Conversations. (His point on accountability starts at 30:34.)
From a business perspective, you may want to highlight the fact that unit tests can "de-risk" any changes you make to your code. Once you have a suite of unit tests, you can make changes to the code base and know what breaks and what doesn't.
I might not be a bad idea to go over user testing. If you have a good set of tests, you can bring failing tests to the users after you make changes to have them validate that the new results are correct. Additionally, you can streamline requirements gathering if you have the users write new unit test definitions for you. They don't need to be able to code, but they do need to be able to give you the appropriate inputs and expected outputs (otherwise how would they know if the changes they asked for were working?).
Visual Studio has a pretty nice set of tools for unit testing, so an example or two may go a long way toward giving your group an idea of what unit testing is like in practice.
Wear this t-shirt ;-)
Well prepared live demo:
Find a "bug" in your "application"
Write a unit test that covers this bug.
Fix this bug
Show, your code is green.
So you can prove, that there's no way, that this bug will occur once more!
Another way to do this:
Propose an issue that can be solved by creating an algorithm. Something relatively simple, of course. Next, code this algorithm in a DLL project. Try to sneak in some weaknesses (i <= array.Length is always a good one). Next, ask them how they would test this DLL.
Most devs run their apps to test them. But you can't run a DLL. You might get some suggesting you create a console app to create methods that exercise the algorithm. Show them how you can craft unit tests to do this.
Have a good set of resources for follow up/self directed learning:
the pragmatic unit test for java/c# are good books on the subject
the kent beck paper on unit testing
links to any larger samples using the testing framework of choice
Unit testing is, roughly speaking, testing bits of your code in isolation with test code. The immediate advantages that come to mind are:
Running the tests becomes automate-able and repeatable
You can test at a much more granular level than point-and-click testing via a GUI
Rytmis
My question is, what are the current "best practices" in terms of tools as well as when and where to use unit testing as part of your daily coding?
Lets try to be somewhat language agnostic and cover all the bases.
Ok here's some best practices from some one who doesn't unit test as much as he should...cough.
Make sure your tests test one
thing and one thing only.
Write unit tests as you go. Preferably before you write the code you are testing against.
Do not unit test the GUI.
Separate your concerns.
Minimise the dependencies of your tests.
Mock behviour with mocks.
You might want to look at TDD on Three Index Cards and Three Index Cards to Easily Remember the Essence of Test-Driven Development:
Card #1. Uncle Bob’s Three Laws
Write no production code except to pass a failing test.
Write only enough of a test to demonstrate a failure.
Write only enough production code to pass the test.
Card #2: FIRST Principles
Fast: Mind-numbingly fast, as in hundreds or thousands per second.
Isolated: The test isolates a fault clearly.
Repeatable: I can run it repeatedly and it will pass or fail the same way each time.
Self-verifying: The Test is unambiguously pass-fail.
Timely: Produced in lockstep with tiny code changes.
Card #3: Core of TDD
Red: test fails
Green: test passes
Refactor: clean code and tests
The so-called xUnit framework is widely used. It was originally developed for Smalltalk as SUnit, evolved into JUnit for Java, and now has many other implementations such as NUnit for .Net. It's almost a de facto standard - if you say you're using unit tests, a majority of other developers will assume you mean xUnit or similar.
A great resource for 'best practices' is the Google Testing Blog, for example a recent post on Writing Testable Code is a fantastic resource. Specifically their 'Testing on the Toilet' series weekly posts are great for posting around your cube, or toilet, so you can always be thinking about testing.
The xUnit family are the mainstay of unit testing. They are integrated into the likes of Netbeans, Eclipse and many other IDEs. They offer a simple, structured solution to unit testing.
One thing I always try and do when writing a test is to minimise external code usage. By that I mean: I try to minimise the setup and teardown code for the test as much as possible and try to avoid using other modules/code blocks as much as possible. Well-written modular code shouldn't require too much external code in it's setup and teardown.
NUnit is a good tool for any of the .NET languages.
Unit tests can be used in a number of ways:
Test Logic
Increase separation of code units. If you can't fully test a function or section of code, then the parts that make it up are too interdependant.
Drive development, some people write tests before they write the code to be tested. This forces you to think about what you want the code to do, and then gives you a definite guideline on when you have acheived that.
Don't forget refactoring support. ReSharper on .NET provides automatic refactoring and quick fixes for missing code. That means if you write a call to something that does not exist, ReSharper will ask if you want to create the missing piece.