Can I invoke nested functions for unit testing? - unit-testing

In F# I want to perform unit testing on a function with several levels of nested functions.
I want to be able to test the nested functions individually as well, but I do not know how I could invoke them.
When debugging, each of these nested functions is invoked as a type of function object, but I don't know if I can access them at compile time.
I do not want to change the nesting scheme that I am using because it makes the most sense functionally to have them nested this way because there is a de facto "inheritance" of some of the function parameters at each nested level.
Is something like this possible? If not, what is the general procedure for unit testing nested functions? Are they tested individually with extra parameters and then inserted into their nested position afterwords never to be able to be tested again?
Very small example:
let range a b =
let lower = ceil a |> int
let upper = floor b |> int
if lower > upper then
Seq.empty
else
seq{ for i in lower..upper -> i}
How could I test that lower or upper are working properly without changing the nested nature of the code?

I would agree with Daniels comment - if the outer function works correctly, you should not need to test any of the inner functions. Inner functions are really an implementation detail that should not be relevant (especially in functional code, where output does not depend on anything else than inputs). In C#, you also don't test whether for loop or while loop inside your method works correctly.
If both the inner and the outer functions are too complex, then perhaps it would be better to write the inner function as a separate function anyway.
That said, you can, of course, mess with the compiled assembly using reflection and invoke the inner function. Inner functions are compiled as classes with constructor that takes the closure (captured values of the outer function) and Invoke method that takes the actual parameters.
The following trivial example works, though I have not tested it on anything more realistic:
open NUnit.Framework
// Function with 'inner' that captures the argument 'a' and takes additional 'x'
let outer a b =
let inner x = x + a + 1
(inner a) * (inner b)
// Unit tests that use reflection in a hacky way to test 'inner'
[<TestFixture>]
module Tests =
open System
open System.Reflection
// Runs the specified compiled function - assumes that 'name' of inner functions
// is unique in the current assembly (!) and that you can correctly guess what
// are the variables captured by the closure (!)
let run name closure args =
// Lots of unchecked assumptions all the way through...
let typ =
Assembly.GetExecutingAssembly().GetTypes()
|> Seq.find (fun typ ->
let at = typ.Name.IndexOf('#')
(at > 0) && (typ.Name.Substring(0, at) = name) )
let flags = BindingFlags.Instance ||| BindingFlags.NonPublic
let ctor = typ.GetConstructors(flags) |> Seq.head
let f = ctor.Invoke(closure)
let invoke = f.GetType().GetMethod("Invoke")
invoke.Invoke(f, args)
/// Test that 'inner 10' returns '14' if inside outer where 'a = 3'
[<Test>]
let test () =
Assert.AreEqual(run "inner" [| box 3 |] [| box 10 |], 14)

Related

Long functions in OCaml without where clauses

What is the idiomatic way to write the following code in OCaml, with better readability?
let big_function arg =
let big_helper_fn acc = function
| p -> ...
...
...
... foo(arg)
...
...
| _ -> ...
in
let small_helper_1 a b =
...
...
in
let small_helper_2 a b =
...
...
in
fold big_function default_acc
%> small_helper_1 aa1
%> small_helper_2 aa2
Hoisting the inner functions outside may be undesirable for two reasons:
One may need to pass several arguments explicitly instead of direct access (shown with foo(arg) above). This would become cumbersome if there are more arguments, say big_function takes in 3 arguments, big_helper_fn uses all of them and the accumulator is a tuple of 3 elements too.
The helper functions become unnecessarily visible in a larger scope than needed. They might be distracting when one is simply skimming the modules because of the same indentation depth with the important big_function.
If OCaml had a first class where clause this wouldn't be a problem. I did find a PPX and another Github repo for this though.
Please provide a reference to a book/style guide/official documentation/name of a large project which follows the method suggested in your answer.
Edit: The trouble I'm having with this code example is that readability is impaired as the actual definition of big_function is separated by a lot from the original let big_function ... statement at the top. So I'm looking for a more readable idiomatic alternative.
It's hard to tell without knowing what's going on in the helper functions, but generally speaking you can extract them as top-level functions. This has several advantages:
it is easier to read
it reduces the number of variables in scope at every point, reducing risks of referring to the wrong variable (can happen with folds/accumulators)
it makes it possible to test inner functions
There are some disadvantages as well, as you said:
there will be more toplevel functions, so there can be naming conflicts (you can use a module to help with that)
you need to pass more variables explicitly instead of by closures. I'd say that this is an advantage, as this makes coupling more obvious. This is an opportunity to pass less data around (say, a field instead of the whole record).
Your code looks already very OCamlish. Many large projects are written in such a way: see OCaml compiler implementation itself. I like where in Haskell, but personally I am against arbitrary where in non pure language since it may make the ordering of side effects very confusing, which may result into bugs very hard to fix. Restricting definitions of where only to non expansive expressions would be ok but I am not sure the PPX you mention performs such a check or not.
I have never done this, but you could put the "main" task first by using let rec:
let big_function arg =
let rec go () =
fold big_helper_fn default_acc
%> small_helper_1 aa1
%> small_helper_2 aa2
and big_helper_fn acc = function
..
and small_helper_1 a b =
..
and small_helper_2 a b =
..
in
go ()
Or, you could use a local module:
module BigFunctionHelpers(A : sig val arg : t end) = struct
open A
let big_helper_fn acc = function ... foo(arg) ...
let small_helper_1 a b = ...
let small_helper_2 a b = ...
end
let big_function arg =
let module H = BigFunctionHelpers(struct let arg = arg end) in
let open H in
fold big_helper_fn default_acc
%> small_helper_1 aa1
%> small_helper_2 aa2
I do this sometimes when extracting local definitions with many parameters from the parent function.

Passing discriminated unions to InlineData attributes

I am trying to unit test a parser that parses a string and returns the corresponding abstract syntax tree (represented as a discriminated union). I figured it would be pretty compact to use Xunit.Extensions' attribute InlineData to stack all test cases on one another:
[<Theory>]
[<InlineData("1 +1 ", Binary(Literal(Number(1.0)), Add, Literal(Number(1.0))))>]
...
let ``parsed string matches the expected result`` () =
However, compiler complains that the second argument is not a literal (compile time constant if I understand it correctly).
Is there a workaround for this? If not, what would be the most sensible way to structure parser result tests while keeping every case as a separate unit test?
One possibility is to use xUnit's MemberData attribute. A disadvantage with this approach is that this parameterized test appears in Visual Studio's Test Explorer as one test instead of two separate tests because collections lack xUnit's IXunitSerializable interface and xUnit hasn't added build-in serialization support for that type either. See xunit/xunit/issues/429 for more information.
Here is a minimal working example.
module TestModule
open Xunit
type DU = A | B | C
type TestType () =
static member TestProperty
with get() : obj[] list =
[
[| A; "a" |]
[| B; "b" |]
]
[<Theory>]
[<MemberData("TestProperty")>]
member __.TestMethod (a:DU) (b:string) =
Assert.Equal(A, a)
See also this similar question in which I give a similar answer.

unit test private methods in F#

Let's say we have a class
type ThisClassIsComplicated () =
let calculateSomething a b =
a + b
In this case calculateSomething is trivial, but if it would be more complicated it may make sense to verify that the calculations done there are correct.
It might make sense to use a unit testing framework to test that private methods.
My question: how to unit test private methods in F#?
Some random thoughts:
The selected answer here, suggests to use the InternalsVisibleTo attribute which anyway is applicable only to internalmethods.
What is the route specific to F# if any? Is this better in a F# design?
let calculateSomething a b = a + b
type ThisClassIsComplicated () =
member this.Calculate a b = calculateSomething a b
Maybe the scope of calculateSomething could be even narrowed down by having a nested module.
If you feel like your code is too complicated to test it from the outside, use the latter option. And in case you want to test an inner function like
let myComplicatedOperation input =
let calculateSomething a b =
a + b
calculateSomething (fst input) (snd input)
you can always rewrite it with currying like this:
let myComplicatedOperation calculateSomething input =
calculateSomething (fst input) (snd input)
Your question does not seem to be directly related to F# though. The general way to test private methods is typically by extracting a class (or, in F#, you can also just extract a let bound function). And making your testee public on that other class / function.
I think that loosening access restrictions in a class/module to facilitate testing is often a bad idea. If you have decided something is irrelevant to know for the outside world, you wanting to test it doesn't make it any less irrelevant.
Can't you just have a public method/function in your class/module that does the testing?
type ThisClassIsComplicated () =
let calculateSomething a b =
a + b
member private this.TestInstance () =
printfn "%A" <| calculateSomething 1 2
static member Test () =
(new ThisClassIsComplicated()).TestInstance()
You can use Impromptu Interface to invoke private methods.
For example, I test the function calcNodeLabel at
https://code.google.com/p/fseye/source/browse/trunk/FsEye/Forms/WatchTreeView.fs#73 like so: https://code.google.com/p/fseye/source/browse/trunk/Test.FsEye/WatchTreeViewLabelCalculatorTests.fs#54
But you need to be careful testing hidden functions in F#: it's an implementation detail of the compiler how the function will actually be compiled (e.g. as a method, as a delegate, as a ...).
Folks will warn generally against testing private methods, but I think it is a bit simplistic to say "never test private methods", since such a declaration takes for granted that access levels as specified in the .NET framework are the only way they could be.
For example, calcNodeLabel in my example should indeed be hidden from the great wide world, but I would consider it part of the internal contract of the class. Of course, you could argue that the class view data and the view itself should be separated, but the point stands: all models are imperfect!

Checking function equality in a F# unit test

I have a bunch of F# functions that implement different algorithms for the same input, kind of like the Strategy pattern. To pick the right strategy, I want to pattern match on the input argument and return the function as a value :
let equalStrategy points : seq<double> =
...
let multiplyStrategy factor (points: seq<double>) =
...
let getStrategy relationship =
match relationship with
| "=" -> equalStrategy
| "*5" -> multiplyStrategy 5.0
| _ -> raise (new System.NotImplementedException(" relationship not handled"))
Now I want to write some unit tests to make sure that I return the right strategy, so I tried something like this in nUnit :
[<TestCase("=")>]
[<Test>]
member self.getEqualstrategy( relationship:string ) =
let strategy = getStrategy relationship
Assert.AreEqual( strategy, equalStrategy )
Now I think the code is correct and will do what I want, but the assertion fails because functions don't seem to have an equality operation defined on them. so my questions are :
(a) is there a way to compare 2 functions to see if they are the same, i.e. let isFoo bar = foo == bar, that I can use in an nUnit assertion?
or
(b) is there another unit testing framework that will do this assertion for me in F#?
Testing whether an F# function returned by your getStrategy is the same function as one of the funcions you defined is also essentially impossible.
To give some details - the F# compiler generates a class that inherits from FSharpFunc when you return a function as a value. More importantly, it generates a new class each time you create a function value, so you cannot compare the types of the classes.
The structure of the generated classes is something like this:
class getStrategy#7 : FSharpFunc<IEnumerable<double>, IEnumerable<double>> {
public override IEnumerable<double> Invoke(IEnumerable<double> points) {
// Calls the function that you're returning from 'getStrategy'
return Test.equalStrategy(points);
}
}
// Later - in the body of 'getStrategy':
return new getStrategy#7(); // Returns a new instance of the single-purpose class
In principle, you could use Reflection to look inside the Invoke method and find which function is called from there, but that's not going to be a reliable solution.
In practice - I think you should probably use some other simpler test to check whether the getStrategy function returned the right algorithm. If you run the returned strategy on a couple of sample inputs, that should be enough to verify that the returned algorithm is the right one and you won't be relying on implementation details (such as whether the getStrategy function just returns a named function or whether it returns a new lambda function with the same behaviour.
Alternatively, you could wrap functions in Func<_, _> delegates and use the same approach that would work in C#. However, I think that checking whether getStrategy returns a particular reference is a too detailed test that just restricts your implementation.
Functions doesn't have equality comparer:
You will have error: The type '('a -> 'a)' does not support the 'equality' constraint because it is a function type
There is a good post here
It would be very difficult for the F# compiler to prove formally that two functions always have the same output (given the same input). If that was possible, you could use F# to prove mathematical theorems quite trivially.
As the next best thing, for pure functions, you can verify that two functions have the same output for a large enough sample of different inputs. Tools like fscheck can help you automate this type of test. I have not used it, but I've used scalacheck that is based on the same idea (both are ports from Haskell's QuickCheck)

Clojure Style Agents in F#

I'm trying to code some Clojure style Agents in F# using MailboxProcessors. Here's what I have so far:
namespace GameEngine
type Agent<'T>(inital:'T) =
let mutable state:'T = inital
let queue = new MailboxProcessor<'T -> 'T>( fun inbox ->
let rec loop count =
async {
let! msg = inbox.Receive()
state <- msg(state)
return! loop(count + 1)
}
loop 0)
do
queue.Start()
member self.Send(action:'T -> 'T) =
queue.Post(action)
member self.Deref() =
state
So the basic idea is that we have a mutable state that can be updated by calling .Send(). My question is, will my messages ever be out of order? If msg A is sent before B will the async function above always process A before B?
Seems like there should be a class like this already in F#? Am I reinventing the wheel?
If msg A is sent before B will the async function above always process A before B?
Yes. (You can see the code for Mailbox
http://fsharppowerpack.codeplex.com/SourceControl/changeset/view/54799#970072
browse to compiler\2.0\Nov2010\src\fsharp\FSharp.Core\control.fs, and eventually see e.g.
member x.Post(msg) =
lock syncRoot (fun () ->
arrivals.Enqueue(msg);
...
which shows it's just a queue under a lock.)
Seems like there should be a class like this already in F#? Am I reinventing the wheel?
Well, it's not immediately clear to me how this is different from just updating a mutable global variable willy-nilly (modulo atomicity of simultaneous updates; you said "before" in the question, so I am unclear if that aspect matters to you). What's the context for wanting this?
There is no built-in implementation of the Clojure-style Agent.
I also at one point worked up a quick and dirty F# implementation similar to yours, but did not take the time to consider all the correctness issues involved; in particular, is it not true that 'T may be a value type (a struct) larger than 64 bits (or 32 bits as the case may be) which could cause a tear (I presume that Clojure like Java doesn't have structs to worry about here). Perhaps an F# generic type constraint ('T when 'T : not struct) would be needed?