is there sample/open_source application for download that is covered with tests(integration, unit, ...) and could be example how testing should be done? We want to learn our junior programmers how tests should be written(ok ok, and we(seniors) want to learn, too :))
Thanx
I would bet the JUnit framework is well covered with unit tests :-)
You can download the sources from here .
Have a look at public metrics of sonar. It scans a lot of open-source projects. Just pick the ones with the highest coverage ;)
Coverage list open-source projects
Related
I realize there have been a LOT of questions on this topic but somehow I haven't found one that addressed my needs.
My team is looking to start automated Unit testing our application. We have never done it before and nobody on the team has much experience with it. I have been asked to research and find a framework for automating our Unit Testing. So far, I have narrowed the choices down to MSTest, NUnit and xUnit.
All across the internet, I read negative reviews about MSTest but it seems the most convenient tool to me for our purpose.
1) Our application is .NET
2) We use licensed VS IDE for our development across the team.
3) Source Control is Team Foundation Server 2010
4) We plan to integrate CI/CD as part of our software delivery process.
I feel that MSTest integrates right into this setup and provides the cleanest interface for us to work on. Is there a significant advantage with NUnit or xUnit over MSTest that we should consider?
I am biased because I work on NUnit, but the advantage of NUnit or xUnit is that both frameworks offer more functionality like data driven tests, parallel execution and a host of advanced features.
That said, there is nothing wrong with MSTest especially if your team is not very experienced with unit testing. It is well integrated with Visual Studio and with TFS, so adding tests is fairly painless. Both NUnit and xUnit integrate well too, but they may require more setup.
Go ahead and start with MSTest. It is an easy introduction to unit testing. If you start running into some of the limitations of MSTest, then it is a fairly easy migration to NUnit or xUnit. For simple cases, it is just a matter of changing your NuGet packages and then find/replace the attributes.
When it comes to choosing between xUnit and NUnit, both are great choices but each has a different philosophy. NUnit tends to be more flexible and allows for a range of testing styles and can be used for integration tests. It is however flexible enough to allow you to shoot yourself in the foot and allows you to write tests that purists would frown upon. NUnit leaves the choice up to you.
xUnit however tends to be a bit more opinionated and pushes you towards the 'pit of success' with unit testing. In my experience, it is great for green field projects, but it can make testing some poorly architected brown-field projects difficult without refactoring.
In the end, you can't go wrong with any of these test frameworks.
I would like to add one negative aspect of XUnit.
It does not have even a basic documentation, nothing beyond "Get Started". No API, nothing.
Here is a brief Analysis of the 3 Frameworks and ranked by color.
Legend of the Coloring:
MSTest it is good only because it is simple and integrated into Visual Studio and it comes from Microsoft. Its not flexible enough and not very extensible. But more than just a unit test framework, it also supports Integration tests, Web (load) performance tests. The other 2 are pure Unit Test Frameworks. So I would say MSTest is "Jack of all Trades, Master of none."
NUnit is my choice because its been the best consistently long enough. It is more flexible and simple with very good user base, documentation and community support.
XUnit is least of my choice because it follows a slightly different style and personally I do not see much advantage over NUnit. In some cases you will need to do some refactoring your code to fit into the test cases. But this is just my opinion as there are many who prefers XUnit.
I have listed down the comparisons in two categories:
Framework:
Implementation:
The source of these comparisons
The reference app distributed with MVC3 RTM is a great learning tool in that it shows a comprehensive application. but what app (these days) exist without Unit Tests? It doesn't seem possible that the team who poured so much work into providing such a beast wouldn't have employed Unit Tests as they built it. Why not make that work public?
thx
Believe it or not that's how it is. Not a single unit test :-)
I need to write a unit test suite for a project I am developing in my spare time. Being a CL newbie I was overwhelmed by the amount of choices for a CL implementation, I spent quite some time to choose one. Now I am facing exactly the same thing with unit test frameworks.
A quick glance at http://www.cliki.net/test%20framework shows 20 unit test frameworks! Choice is good but for a novice like me this can be a bit confusing and given the number of frameworks it would be painful to try them all.
I would like to use a framework which:
Is reasonably well maintained
Easy to use but with some degree of flexibility
Offers some sort of integration with Emacs (or it is possible to easily integrate it with Emacs)
It is possible to integrate it with git post-commit hooks
It is possible to integrate it with a continous integration system (such as buildbot)
What are your experiences in this field?
Did you see the link to http://aperiodic.net/phil/archives/Geekery/notes-on-lisp-testing-frameworks.html off the Test framework comparison link on that cliki page you mention? Phil gives his impressions, and what it looks like to use the various test frameworks.
I personally prefer lisp-unit. It's simple to use and has most of the common types of tests.
http://www.cliki.net/lisp-unit
http://repo.or.cz/w/lisp-unit.git/blob_plain/master:/documentation/lisp-unit.html
I don't think it has any integration with post-commit hooks or buildbot built in.
I've recently decided to start using some light unit testing to see if it adds any value to our project, but I'm having trouble finding documentation for SimplyVBUnit. Any suggestions?
There's some discussion of VB6 unit testing in this question, with Gutzofter's mini tutorial on SimplyVBUnit.
If you downloaded the binary installer for SimplyVBUnit it comes with a couple of example projects in the installation folder. Both the source code zip file and binary installer include a Help.chm showing many of the classes and how to apply them in your tests.
Have you considered collecting test coverage? This determines what part of your code has run, often as exercised by your unit tests. To do this, you need a test coverage data collection tool, as well as your tests.
EDIT: The only VB6 test coverage tool I know is http://www.aivosto.com/vbwatch.html
Aivisto seems to have a generally good reputation for thier VB tools.
What is currently the best tool for JavaME unit testing? I´ve never really used unit testing before (shame on me!), so learning curve is important. I would appreciate some pros and cons with your answer. :)
I think it will depend on what kind of tests are you planning to do. Will you be using continuous integration. Is running tests on handsets a must.
If tests are more logic/data processing tests, the you can do fine with JUnit. But if you need to use some classes from javax.microedition.*, then the things will become a bit tricky, but not impossible.
Some examples: a test for text wrapping on screen, that would need javax.microedition.lcdui.Font. You can't just crate a font class using jars shipped with WTK, because at initialization it will be calling some native methods, that are not available.
For these kind of tests I have created a stub implementation of J2ME. Basically these are my own interpretation of J2ME classes. I can set some preconditions there (for example every character is 5 pixels wide, etc). And it is working really great, because my test only need to know, how J2ME classes respond, not how they are internally implemented.
For networking tests I have used MicroEmulator's networking implementation, and it has also worked out well.
One more issue with unit tests - it is better to have your mobile projects as a java project using Java 4,5,6, because writing test in 1.3 is, at leas for me, a pain in the...
I belive, that starting with JUnit will be just fine, to get up and running. And if some other requirements come up (running tests on handsets), then You can explore alternatives.
I'll be honest, the only unit tester I've used in Java is JUnit and a child project for it named DBUnit for database testing... which I'm assuming you won't need under J2ME.
JUnit's pretty easy to use for unit testing, as long as your IDE supports it (Eclipse has support for JUnit built in). You just mark tests with the #Test annotation (org.junit.Test I think). You can also specify methods that should be run #Before or #After each test, as well as before or after the entire class (with #BeforeClass and #AfterClass).
I've never used JUnit under J2ME, though... I work with J2EE at work.
Never found an outstanding one. You can try to read this document :
how to use it
and here the link to : download it
Made by sony ericsson but work for any J2ME development.
And I would recommend you spend some time learning unit testing in plain Java before attacking unit testing on the mobile platform. This may be a to big to swallow one shoot.
Good luck!
There's a framework called J2MEUnit that you could give a try, but it doesn't look like it's still being actively developed:
http://j2meunit.sourceforge.net