For those of us who like to learn by reading good code, what are some of the projects you've seen which provide a great example of a medium to large suite of unit tests in action?
This isn't a question about what your favourite unit testing framework is, but it could be helpful to add which unit testing and/or mocking frameworks are used by the projects you mention.
Any platform will do, but I'm mainly interested in projects which use an xUnit-style unit testing framework in .NET.
Apache CXF has a mind-numbingly large array of unit tests. They are written to Junit, and include the JUnit-Spring mechanisms as well as one of the mock libraries. They include launching processes and many other mechanisms you might need some day.
For example, ReportLab uses Python's unittest module (also known as PyUnit) and has lots unit test that you might want to take a look at.
I'm also using unittest myself. It is very similar to Java's jUnit framework and thus very handy for writing tests in no time.
The Service Factory has about 780 unit tests, including some that test code meant to run inside of Visual Studio. It has some very good examples of mocking such code.
The Enterprise Library has even more.
NunitLite has a unit tests suite but I'm not sure it qualifies as a medium-sized project. I seem to remember NUnit also has unit tests, but its site currently is unavailable.
Related
I was doing TDD using NUnit. I was naming my NUnit tests in a behavioral style (like given, when, then). However I am now using MSpec for all my unit tests. I'm still writing tests first, using mocks, etc... so, they're still unit tests. But, I don't see a need for NUnit.
I am nervous to throw away all the effort I put into learning NUnit. Should I abandon TDD/NUnit completely, taking into consideration that BDD is TDD done right?
Now that you have embraced BDD you are following an "Outside-In" development approach.
A nice succinct definition of this development technique can be found at programmers.stackexchange.com. I quote:
"Outside-In (London school, top-down or "mockist TDD" as Martin Fowler
would call it): you know about the interactions and collaborators
upfront (especially those at top levels) and start there (top level),
mocking necessary dependencies. With every finished component, you
move to the previously mocked collaborators and start with TDD again
there, creating actual implementations (which, even though used, were
not needed before thanks to abstractions). Note that outside-in
approach goes well with YAGNI principle."
When using BDD, you develop in a top-down manner and mock dependencies to satisfy your test. Once your BDD test passes you then revert to using TDD to implement concrete versions of the dependencies you encountered during your BDD test (using an "Inside-Out" approach).
Hence both your TDD and BDD tests are valuable, as they test different aspects of your code i.e. the BDD tests ensure that a user's interaction is tested against all of the layers in your system, whilst the TDD tests cover the individual components in detail and in isolation (via mocking).
So don't abandon your NUnit tests!
To end my answer, you say that:
BDD is TDD done right
As I've explained above, the major difference between BDD and TDD is the scope of code which they cover. Dan North has a good article on this here.
NUnit and MSpec are, basically, test frameworks. They can both be used to write unit, integration, or acceptance tests. You implement the test at the right intersection of layers, behaviors, or whatever your definition is. Both frameworks support BDD-style and naming. MSpec does it up front with the custom delegates. NUnit makes it a little more challenging (you have to fiddle with constructors and setup & test methods).
You're still writing tests first (TDD), but now you're using a test framework that directly supports context/specification-grammar and behavioral testing (BDD) vs. object-structure testing.
The question isn't really about TDD vs. BDD, Arrange-Act-Assert grammar vs. context/specification-grammar, or any of the other structural differences in the test framework (one setup per context, one assertion per spec, etc), but of your skills with a particular framework!
I say, embrace your new knowledge! Do you like mspec? Are you likely to engage your colleagues to switch to mspec? Will you completely forget your NUnit skills (the API or the command-line runner)?
If you inherit some old projects or have team-members who like NUnit, the two frameworks can exist side-by-side in your solution and in your build script with little trouble. It's just not great to have many different ways to write tests and report results.
From my experience there are some cases where NUnit is still a good choice. For example, mspec currently does not support examples, whereas NUnit has TestCase and TestCaseSource. These are useful in Unit Testing scenarios, so there might still be a use for xUnit style tools. No need to "forget" anything, I think it good to be aware of all the tools in your toolbelt and choose the right one for the task at hand.
As a hobby project I'm developing a small Bash unit testing framework. Now that the basic features are implemented the next step would be to write tests for it.
But how do I test my framework using the framework itself?
I've tried to Google an answer to this question but wasn't able to find anything about this subject.
The idea I've come up with so far is writing tests for some trivial things which make use of all the features of the testing framework itself. If the tests are all green, then fine. These basic test could also serve as some sort of documentation.
But I don't know if this is actually a good approach.
The problem will be to test that tests can fail. Unit tests should all be "green". So choosing some tests that need to fail to prove it right is not an option.
I think that the solution will be quiet technology based. It is hard to give general advise about "unit testing a test framework". Which parts belong to the framework? A part that reads files that contain unit tests, a part that runs them, a part that writes some report, a library of helpers, like asserts, ... ?
You need to split these parts and make it possible to run them in isolation. Then it is no difference as any other tests.
Probably you find a way to run the whole testrunner within unit tests. This wouldn't be unit tests as such, but rather integration tests. This kind of tests may become a maintenance problem, because they are usually very complex, use the file system and other stuff from the operation system which shouldn't be used by unit tests. Consider if this is worth the effort.
There is always a part that can't be tested. The part that puts everything together and runs the application (whatever it is) in a real environment. There is no difference here. It is even easier, because some of the "end user environment" can be tested by "green" tests.
I'm in the middle of implementing a unit test infrastructure for a large C++ project, and due to political reasons I'm almost sure CppUnit will be pushed as the unit testing framework.
I'm trying to identify mock frameworks that blend with CppUnit. I've found mockpp, and I've heard that Google Mock should work.
What frameworks work alongside CppUnit?
Mocking libraries are typically independent of the unit testing framework. They accomplish two different jobs, and frankly don't have much reason to talk to each other. Where they do integrate is in answering these questions:
When should I create my mock objects?
When should I initialize the mock objects with my expectations?
When should I validate the mocks were called as expected?
And you do that at the appropriate points in your tests.
For one example, check opmock. http://sourceforge.net/projects/opmock/
According to their wiki, opmock is easily called from a CppUnit test. See http://sourceforge.net/p/opmock/wiki/Using%20Opmock%20with%20other%20unit%20testing%20frameworks/
I have been tasked with finding the best way to do integrated unit testing. We have a very large Java EE 5 application (desktop). Right now we use a tool called QF-TEST which is pretty cumbersome for large tests and can be difficult to use (easy to break) with any code changes.
We now want to do something that is more standard and gives the developers more control.
I have read a few posts here:
Unit testing in Java EE environment
Automated Testing - kind of cool, although for .Net
Best practice approach for automated testing
easiest Automated testing tool in Java
From the general information I have read JUnit/JUnitEE is probably the best (by best I mean quickest to learn and possibly the JAVA standard).
Is JUnit the way to go for large Java EE applications? What are some other options that others find better (if there are any)?
Thanks!
That's a very broad question. It's the topic of many books. Start with JUnit, start reading about test-driven design/development (TDD), and build from there. Ask more specific questions as you come across them. You could start with "Test Infected", a rather old, yet still applicable article on the JUnit site.
I think for java I would go with either with junit or testNG frameworks, If you have persistence/data base involved in the application, I would add dbUnit to the mix.
If you have build scripts like maven or ant or gradle, I would also suggest you look into Jenkins and similar tools to automate builds.
I am suggesting maven because it has life cycle events for testing, You can add targets with ant, I have not used gradle but if I was given choice, I would choose gradle
JUnit is not the way to go for large Java EE applications, it is a bare minimum.
But since you already have an application writing jUnit tests will not help you, because jUnit is not a testing tool. jUnit is used for TDD and refactoring safety, writing jUnit tests as an afterthought simply does not work.
That said you should learn jUnit4 as soon as possible (it can be done in 60 seconds)
and than learn about TDD (it can be done in 2-3 hours)
and use jUnit for every future code change you make to your application.
As for code you already have, what you need is 'code analyzers'.
That means tools that will check your source code for violated coding standards, code duplication, do code coverage, dependency analysis and complexity monitoring.
(All of these and more exist as eclipse plugins)
You will probably also need some 'java gui testing' framework (to replace QF-TEST)
There are a lot of open source alternatives there, search the web.
I've been reading a lot about Unit tests and Test Driven developemnt.
Recently, I also read java unit test code.
I however, prefer to develop in Qt. So I googled up "unit testing in c++" and found a host of information about various unit testing frameworks available for C++.
However, I could not find a reliable comparison of the the various frameworks.
So I look to the SO community to guide me through the selection of what may the "best" unit testing framework for c++.
Also, if anybody had specific comments regarding TDD in Qt (especially using Qt-Creator), then they are more than welcome.
Usually use Boost, but if you are using Qt, their QtTestLib might be the better choice.
I would recommend doctest (created by me) - it's the lightest on compile times from all the popular testing frameworks. It is also a direct competitor to Catch which is currently the most used framework - checkout the differences in the FAQ
This seems too be the same question as:
Unit testing in C++ which is actually c++ despite the URL title.
From there, they link to two more SO questions which should help:
Unit testing for C++ code - Tools and methodology
C++ unit testing framework
There is a table comparing all (?) the C++ unit test frameworks available from wikipedia.
There also is an old comparison of C++ unit test frameworks available. I do not think it has not been updated so I mention it as a complement as it's more argumented than the table. It covers, CppUnit, CppUnitLite, Boost.Test, NanoCppUnit, Unit++, CxxTest, especially it does not cover Google C++ framework.
The "xUnit" family of testing frameworks is usually pretty solid (jUnit, NUnit, etc.). I haven't used it myself, but there is a port of jUnit for C++:
http://sourceforge.net/projects/cppunit
Boost is usually a good choice, and it contains a testing framework, the Boost Test Library. I have used it for small test cases and it did what I expected, but I haven't used it extensively like in TTD.
If you want to get off the ground quickly without figuring out how to build a library, there is a single header file include solution, which supports fixtures (setup and teardown), the usual TEST() {} with CHECK_TRUE, etc.
It also has memory leak detection and performance testing capabilities.
https://gitlab.com/cppocl/unit_test_framework