I use typeclasses, from libraries and my own, extensively in my Purescript code. Each typeclass law seems to provide a nice test. Currently, I'm writing tests individually for each class and law. Is there a way to partially automate this? Maybe something like Haskell's quickcheck-classes, which I, admittedly, haven't tried.
There are tests for the common core classes here: https://github.com/garyb/purescript-quickcheck-laws
I need to break out the various laws so they can be used independently too ideally, but maybe this will suit your needs anyway!
Related
As part of my lecture in C++ the students will have to solve assignments. Each solution shall implement the same functions with the same functionality and the same parameters (function name, return value, passing parameters). Only the code inside is different.
So i'm thinking about a way to test all solutions (around 30) in an effective way. Maybe the best way is to write a unit test as well as a shell script (or something similar) that compiles each solution once with the unit test and runs it.
But maybe there is a different and much better solution to this problem.
The reason why unit tests are one of the most efficient types of automated testing is because the return of investment is relatively small (compared to other types of testing), so it makes perfect sense to me to write a verification suite of tests.
You might even go so far as to give the students the test suite instead of a specification written in prose. This could introduce them to the concept of Test-Driven Development (although we normally tend to write the tests iteratively, and not in batches).
yes, unit tests are the obvious solution for most cases.
compiler warnings and static analysis is also useful.
timing the program's execution given a set of parameters is another fairly automated option -- depends on what you are interested in evaluating.
creating base classes with good diagnostics (which you can swap out the implementation for your evaluation if you prefer) is another option. you can also provide interfaces they must use, and hold two implementations. then exercise the programs as usual using the diagnostic implementation. it depends on what you are looking for.
Maybe I'm missing something obvious, but wouldn't it be sufficient just to run the code several times with boundary-testing parameter values?
Recently, i took ownership of some c++ code. I am going to maintain this code, and add new features later on.
I know many people say that it is usually not worth adding unit-tests to existing code, but i would still like to add some tests which will at least partially cover the code. In particular, i would like to add tests which reproduce bugs which i fixed.
Some of the classes are constructed with some pretty complex state, which can make it more difficult to unit-test.
I am also willing to refactor the code to make it easier to test.
Is there any good article you recommend on guidelines which help to identify classes which are easier to unit-test? Do you have any advice of your own?
While Martin Fowler's book on refactoring is a treasure trove of information, why not take a look at "Working Effectively with Legacy Code."
Also, if you're going to be dealing with classes where there's a ton of global variables or huge amounts of state transitions I'd put in a lot of integration checks. Separate out as much of the code which interacts with the code you're refactoring to make sure that all expected inputs in the order they are recieved continue to produce the same outputs. This is critical as it's very easy to "fix" a subtle bug that might have been addressed somewhere else.
Take notes too. If you do find that there is a bug which another function/class expects and handles properly you'll want to change both at the same time. That's difficult unless you keep thorough records.
Presumably the code was written for a purpose, and a unit test will check if the purpose is met, i.e. the pre-conditions and post-conditions hold for the methods.
If the public class methods are such that you can externally check the state it can be unit tested easily enough (black-box test). If the class state is invisible or if you have to test tricky private methods, your test class may need to be a friend (white-box test).
A class that is hard to unit test will be one that
Has enormous dependencies, i.e. tightly coupled
Is intended to work in a high-volume or multi-threaded environment. There you would use a system test rather than a unit test and the actual output may not be totally determinate.
I written a fair number of blog posts about unit testing, non-trivial, C++ code: http://www.lenholgate.com/blog/2004/05/practical-testing.html
I've also written quite a lot about adding tests to existing code: http://www.lenholgate.com/blog/testing/
Almost everything can and should be unit tested. If not directly, then by using mock classes.
Since you decided to refactor your classes, try to use BDD or TDD approach.
To prevent breaking existing functionality, the only way is to have good integration tests, but usually it takes time to execute them all for a complex system.
Without more details on what you do, it is not that easy to give more implementation details. Some are :
use MVP or presenter first for developing gui
use design patterns where appropriate
use function and member pointers, or observer design pattern to break dependencies
I think that if you're having to come up with some "measure" to test if a class is testable, you're already fscked. You should be able to tell just by looking at it: can you write an independent program that links to this class alone and makes sure it works?
If a class is too huge so that you can't be sure just by looking at it...chances are it probably isn't testable. People that don't know how to make small, distinct interfaces generally don't know how to adhere to any other principle either.
In the end though, the way to find out if a class is testable is to try to put it in a harness. If you end up having to pull in half your program to do it, try refactoring. If you find that you can't even perform the most basic refactor without having to rewrite the entire program, analyze the expense of doing so.
We at IPL published a paper It's testing Jim, but not as we know it which explores the practical problems of testing C++ and suggests some techniques to address them that may well be of use given your question. These techniques are also well supported in Cantata++ - our C/C++ unit and integration testing tool.
How does TDD compare with Functional Programming Languages like F# and Erlang?
I haven't actually worked directly with a functional programming language yet, but from what I've seen of it, you have two sides of an equation and they have to balance like in algebra or accounting; this seems somewhat reminiscent of TDD where you define your expected outputs as Assert statements (one side of the equation) and the rest of the functionality goes into a class decoupled from the test (the other side of the equation), except that functional programming IMHO seems a bit cleaner.
Do the two actually have similarities, or am I just overthinking this a bit?
Software Design v Development Methodology
They're orthogonal.
TDD is an approach to developing software which focuses on ensuring correctness by developing tests against specifications before the production code is written. Functional programming is a paradigm for designing and implementing software.
I think TDD and functional programming (FP) are different in that TDD is a methodology and FP is programming paradigm.
I would say that FP help when practicing TDD as FP encourages you to make things deterministic when possible. Deterministic functions are much easier to test than non-deterministic ones.
Chris is correct in saying that they are orthogonal. However, there are some aspects of functional programming that make testing of functional programs a lot easier.
Functional programs are composed from functions and guarantee that the function will behave the same in all contexts. This means that when you test a function in unit test, you know that it will always work this way. You don't have to test whether it works if you plug it into some other environment.
Functions take arguments and return results and that's the only thing they do. This means that you can usually avoid mocking and similar tricks, because you don't need to verify whether a function does some call on some object. You only need to verify that it returns the expected result for given arguments.
Finally, there are some nice automatic tools for testing functional programs. For F#, we have FsCheck (which is based on QuickCheck known from Haskell). These benefit from various properties of functional programs.
So, they both have different purpose and are essentially a different thing, but there are some nice relations (perhaps like a tea and a teapot :-) they are completely different things, but work very well together!)
You're correct that when writing a functional program you might use equational reasoning to derive the definition of a function. However, that reasoning typically doesn't exist in some reified form (such as tests), so it is not generally the case that a function is proven correct in a way that is machine- or human-checkable. It is certainly possible to use TDD with functional languages (e.g. using any .NET compatible TDD library with F#) to verify that functions have been derived correctly, but there are also other testing strategies which might be more unique to functional languages, such as using QuickCheck for randomized specification checking.
I think the similar feel between the two stems from the fact that, with both, functions are supposed to be deterministic. FP functions shouldn't have side effects and side effects in test functions for object orientated code should be removed by injecting stubs.
Does anyone have any references for building a full Object/Class reflection system in C++ ?
Ive seen some crazy macro / template solutions however ive never found a system which solves everything to a level im comfortable with.
Thanks!
Using templates and macros to automatically, or semi-automatically, define everything is pretty much the only option in C++. C++ has very weak reflection/introspection abilities. However, if what you want to do is mainly serialization and storage, this has already been implemented in the Boost Serialization libraries. You can do this by either implementing a serializer method on the class, or have an external function if you don't want to modify the class.
This doesn't seem to be what you were asking though. I'm guessing you want something like automatic serialization which requires no extra effort on the part of the class implementer. They have this in Python, and Java, and many other languages, but not C++. In order to get what you want, you would need to implement your own object system like, perhaps, the meta-object system that IgKh mentioned in his answer.
If you want to do that, I'd suggest looking at how JavaScript implements objects. JavaScript uses a prototype based object system, which is reasonably simple, yet fairly powerful. I recommend this because it seems to me like it would be easier to implement if you had to do it yourself. If you are in the mood for reading a VERY long-winded explanation on the benefits and elegance of prototypes, you can find an essay on the subject at Steve Yegge's blog. He is a very experienced programmer, so I give his opinions some credence, but I have never done this myself so I can only point to what others have said.
If you wanted to remain with the more C++ style of classes and instances instead of the less familiar prototypes, look at how Python objects and serialization work. Python also use a "properties" approach to implementing its objects, but the properties are used to implement classes and inheritance instead of a prototype based system, so it may be a little more familiar.
Sorry that I don't have a simpler answer to your question! But hopefully this will help.
I'm not entirely sure that I understood you intention, however the Qt framework contains a powerful meta object system that lets you do most operation expected from a reflection a system: Getting the class name as string, checking if a object is a instance of a given type, listing and invoking methods, etc.
I've used ROOT's Reflex library with good results. Rather than using crazy macro / template solutions like you described, it processes your C++ header files at build time to create reflection dictionaries then operates off of those.
I want to write a simple colour management framework in C#, Java and AS3. I only want to write the unit tests once though, rather than recreating the unit tests in JUnit, FlexUnit and say NUnit.
I have in mind the idea of say an xml file that defines manipulations of "instance" and assertions based on the state of "instance" via setup, teardown and a set tests. Then to have a utility that can convert that XML into xUnit code, for an arbitrary number of xUnits. Before I start wasting time developing such a solution though, I want to make sure no similar solution already exists.
Would FIT/ Fitnesse be suitable for what you want?
FIT is an acceptance test framework rather than unit test framework, but from what you describe you would want to ensure that the three implementations have the same behavior rather than identical designs.
FIT has links to several languages
I think you are overcomplicating things... you might consider a scripting language that you can use against all 3. I know Ruby could be used to test Java via JRuby, and C# via IronRuby, but I don't know about AS3.
I have never needed to do this myself, but I imagine a dynamic language like Ruby could really let you do it without a lot of extra work.
As a side note, you could also try writing a compiler of sorts, much like FogCreek's (in)famous Wasabi language, then you could write both your code and tests in that language, and have the compiler do your work.... this of course would probably be overcomplicated, but I think it would be a lot better than attempting to define an XML test language... and potentially a lot more readable.
You could also check out Fitnesse with Slim, as Slim should be a lot more lightweight to implement for new languages (AS3). I guess it's more about acceptance/integration testing than unit testing, but it could be worth looking into.