I have a small library that is ported to different platforms (win, os x, ios, android) that I am using in some small personal utilities and I am starting to look into testing more thoroughly.
Right now, I am using NUnit to test the windows version, with test cases written in c#. The setup just basically loads the dll and the test cases make various calls to the library. I want to duplicate this testing on the other platforms. I know there are other xUnit type frameworks to accomplish this on the other platforms, JUnit etc., but I would rather not have to rewrite the test cases for each platform.
Is there a recommended way to have one set of test cases and just point them to one of the libraries to run against that platform?
I still consider myself a novice programmer, so apologies if it's not possible and the question is naive.
Thanks
Due to the nature of unit testing(testing individual procedures or functions rather than overall program functionality as in blackbox testing ), and different languages supports different language specific features, there cannot exist a universal unit testing framework that supports all programming languages (in your case, Java, C#, Objective-C ). That's why we have xUnit framework for different languages.
Even if someone creates such framework, it would not be possible to properly test functions that uses language specific features.
Related
What are the most modern approaches to unit testing for the C++ language? The class of the languages with bigger introspection power (like Python) have unit testing frameworks that are somehow more natural to use. Unit tests can be defined more easily. In comparison, the classical CppUnit (based on JUnit) seems to take very conservative approach. Is there anything newer and better that uses the specific power of C++ (or even C++11) to make the life easier?
I have been using CppUnit framework for a while in rather simplistic way for parts of the project on Windows native C++ (Visual Studio 2005 and 2010). We did not choose the Test Driven Development approach earlier, because there already was a lot of legacy code, and we had found it quite difficult to add tests for it. We had to refactor the application, but adding all the nice tests would be time consuming even in the case.
Recently, we have switched to Visual Studio 2013 (because of the C++11 standard implementation), and we are going to start new, rather long-term project.
Having the previous good (small) experience with unit testing, I would like to try the Test Driven Development approach. As the project is not a tiny one (expected size about the same as the older one, i.e. about 200 k lines of code), I prefer rather more easy (but not less capable) framework.
There is a chance the new project could lead to cross-platform implementation (Windows and Linux). There is a unit testing support in Visual Studio 2013, but I have no experience with it and how would it fit with the cross-platform.
So far, I have found the list of unit testing frameworks for C++. However, one cannot see how they differ in principle. I currently have three candidates (conservative choice):
Boost -- the probable candidate; testbed for C++ standards; hence it is likely it is to be widely accepted; probably the biggest user group. It seems to be more advanced than CppUnit.
CppUnit -- I know it, but it writing all of the code around is not a pleasure.
Visual Studio 2013 built-in -- new to me, somehow can generate the skeletons probably.
Anyway, it seems that all of the three use similar approach. Possibly the VS2013 supports generating the code, but it does not mean it leads to anything simpler.
Is there any radically new approach?
The only test framework worth considering:
Catch
For an introduction to the lib, see also here and here
It's easy to use (a header-only lib consisting of just one header), portable, and has by far the simplest, cleanest syntax of any C++ unit testing framework.
And unlike other libraries, you don't need to remember two dozen different macros f or different types of assertions.
You just use REQUIRE:
int one = 1;
REQUIRE( one == 2 );
which, through some clever operator overloading, will show both the original expression and the expanded argument values in the output:
test.cc(7): FAILED:
REQUIRE( one == 2 )
with expansion:
1 == 43
Compared to this, every other framework is a chore to use IMO.
I used to use Boost.Test before I found this, but that was a lot more cumbersome to set up and use. We use CppUnit at work, and that seems to be designed to be as fragile and painful as possible.
I've looked briefly at the VS2013 test framework, but haven't tried it, and it looks tolerable, but very much like it's emulating "the old guard". It doesn't really try to be cleaner, easier or better than CppUnit, Boost.Test and all the other ones that came before Catch. So I'd say don't bother with it. Tests should be easy to write (and to understand), and Catch is lightyears ahead of every other framework I've seen on that front.
I've been using theVisual Studio 2013 built in test framework for about 6 weeks now and really like it. The integration is excellent and it's very easy to pick up. If you're working on a project that only targets Windows then I thoroughly recommend it.
Is there any way to generate automated unit test cases for c++ code written using VS2005. I do some basic R&D and found that there is a support for automated testing in VS2005 but its only for Managed Code, so i am looking for something specific to:
"Automatic generation of Test cases of Native C++ Code"
It doesn't matter whether its a plug-in that works with VS2005 or any standalone application. But it is preferred to have some plug-in sort of solution.
I can't tell if you are asking about a unit testing framework (if yes, try cppUnit), or if you really want tests code-generated. If the latter, I imagine the answer is no. Usually a capability like that is linked to reflection and/or built-in design-by-contract capabilities (e.g Eiffel or C# with .net 4.0).
I want to know if anyone of you guys use TDD in your c++ projects and how it performs compared to managed languages like C# and Java.
And what frameworks you guys are using to automate tests on c++ projects?
Two useful C++ test frameworks that don't seem to have been mentioned yet are Boost test and Google Test.
Test Driven Development is possible in any language. You need the right testing tools and methodologies for the language, and may possibly need a custom testing infrastructure for your project.
I have found CppUnit (at least 1.x) to be a very poor framework -- it seems to use Java/C# idioms in a C++ language and does not have support for STL constructs.
If you want a good example of Test Driven Development (in C), look at the Wine project -- http://test.winehq.org/data/ shows their test results across the different versions of Windows, Wine and the different commits into the Wine repository. They have their own custom test infrastructure.
I recently moved from a C# project that was developed using TDD to a project that is using C++. I was dreading it quite a bit, but I find that doing C++ with TDD is a lot more enjoyable and the code is more robust than I remember from past (non-TDD) experiences with C++.
We are using Google Test. It is not as easy to use as NUnit/MbUnit, but it seems to work pretty well. There is also a Google mocking framework http://code.google.com/p/googlemock , but I have not been using that yet.
I have an interesting situation where I am refactoring a bunch of ObjC iPhone code to create a C++ API. I'm a novice to C++ and looking into C++ mocking frameworks to augment the work I'd done using OCUnit and poor man's mocks. I ran across googlemock and wanted to know if anyone has ever used it for iPhone development? Also, how can I share this (or mockpp) with other devs as it is an installable package and doesn't seem to lend itself to checking into a repository?
I've never used Googlemock for iPhone development, but have used it plenty on Windows and various UNIXes.
It uses standard modern C++ with TC1 (Technical Corrigenda 1) so can compile on any up to date, compliant compiler.
If your development environment doesn't implement TC1, then Google also include a subset of the Boost library that implements Tuples, which is the part of TC1 that Googlemock depends on.
Basically, if your compiler can handle templates, it should be able to handle Googlemock.
You can download the full source from Googlecode and this is what you might want to check into your repository.
For the Objective-C code you might consider Kiwi (http://www.kiwi-lib.info/mocks_and_stubs.html). It is a great bdd framework with nice mock/stub support.
For C++, there are lots of good unit test frameworks out there, but I couldn't find a good one for functional testing. With functional testing, I mean stuff which touches the disk, requires the whole application to be in place etc.
Point in case: What framework helps with testing things like whether your I/O works? I've got a hand-rolled system in place, which creates temporary folders, copies around a bunch of data, so the tests are always done in the same environment, but before I spend more time on my custom framework -- is there a good one out there already?
I wrote one from scratch three times already - twice for testing C++ apps that talked to exchanges using FIX protocol, once for a GUI app.
The problem is, you need to emulate the outside world to do proper system testing. I don't mean "outside of your code" - outside of your application. This involves emulating end users, outside entities, the Internet and so on.
I usually use perl to write my system testing framework and tests, mostly because it's good with accessing all sorts of OS facilities and regexps are first-class citizens.
Some tips: make sure your logs are easy to parse, detailed but not too verbose. Have a sane default configuration. Make it easy to "reset" the application - you need to do it after each test.
The approach I usually use is to have some sort of "adapter" that turns the application's communications with the outside world into stdin/stdout of some executable. Then I build a perl framework on top of that, and then the test cases use the framework.
Below I list a couple of tools and larger testing applications of which I am aware. If you provide more information on your platform (OS, etc.) we can probably provide better answers.
For part of what you require, Microsoft provides the Application Verifier:
Application Verifier (AppVerifier) is a runtime verification tool used in testing applications for compatibility with Microsoft Windows XP. This tool can be used to test for a wide variety of known compatibility issues while the application is running. This article details the steps for using AppVerifier as an effective addition to the application development and testing cycles.
Application Verifier can be useful for testing out low memory conditions, other low resources, and other API usage.
Another part of the puzzle, is the Microsoft Detours package, which can be used to replace API calls with your own code (useful for say, returning error codes for tests that are hard to set up).
Detours is a library for instrumenting arbitrary Win32 functions on x86, x64, and IA64 machines. Detours intercepts Win32 functions by re-writing the in-memory code for target functions. The Detours package also contains utilities to attach arbitrary DLLs and data segments (called payloads) to any Win32 binary.
There are other, larger (and more expensive) comprehensive packages available too. Borland makes Silk.
Automated Software makes TestComplete. The selection of one of these tools would be up to your needs for your applications.
IBM/Rational provides the Rational Functional Tester, which is available across many platforms, and feature-rich.
Hi I am not sure if the framework we have helps in your situation but it hooks into Rational Functional Tester and allows the user to create various datasets to be attached to different tests and to change the enviornments without changing the scripting and reuses the automation in an efficient way.
Have a look if your interested:
http://www.testpro.com.au/Test-Automation-Framework.html