Should I mock IO or not? - unit-testing

Say I have a simple file based database Monad. I would define it as shown below.
newtype MyDbFileBased a = MyDbFileBased {
unMyDbDbFileBased :: ExceptT MyDbFileBasedError (ReaderT MyDbFileBasedEnv IO) a
} deriving (
Functor
, Applicative
, Monad
, MonadError MyDbFileBasedError
, MonadReader MyDbFileBasedEnv
, MonadIO
)
I have read that the above pattern (will post a link to the blog when I find it) is not recommended and that I should replace IO with a generic Monad like so.
import qualified Data.ByteString as B
newtype MyDbFileBased m a = MyDbFileBased {
unMyDbFileBased :: ExceptT MyDbFileBasedError (ReaderT MyDbFileBasedEnv m) a
} deriving (
Functor
, Applicative
, Monad
, MonadError MyDbFileBasedError
, MonadReader MyDbFileBasedEnv
)
class Monad m => MonadFileBasedIO m where
readBytes :: FilePath -> m B.ByteString
writeBytes :: FilePath -> B.ByteString -> m ()
...
instance MonadFileBasedIO IO where
readBytes = B.readFile
writeBytes = B.writeFile
This will supposedly make unit testing easier. The advise is to mock IO in testing with something like as follows.
data MockFS = EmptyDir
| SingleFile FilePath String
deriving (Show)
newtype MockFileBasedIO a = MockFileBasedIO {
unMockFileBasedIO :: State MockFS a
} deriving (
Functor
, Applicative
, Monad
, MonadState MockFS
)
instance MonadFileBasedIO MockFileBasedIO where
readBytes pathReq = do
dir <- get
case dir of
EmptyDir -> fail "file not found"
SingleFile path contents -> if pathReq == path
then pure (BU.fromString contents)
else fail "file not found"
writeBytes path = put . SingleFile path . BU.toString
All this looks good to me until now. But then I want to add things like catch and liftIO to functions within MyDbFileBased type. I thought of adding catchMonadFileBasedIO and liftMonadFileBasedIO functions to MonadFileBasedIO typeclass and set catchMonadFileBasedIO = catch and liftMonadFileBasedIO = liftIO for IO monad. But then it brings in dependency on MonadIO and Exception typeclasses and the compiler tells me to add these typeclasses to function signatures of catchMonadFileBasedIO and liftMonadFileBasedIO. Also I would need to derive MonadIO from MyDbFileBased m. Then what's the point of replacing IO with a generic Monad in the first place?
I don't understand if I should be mocking IO in cases such as these or not. How do I use liftIO and catch if we mock it? Should I not catch exceptions in this module and cascade them to application level?

Then what's the point of replacing IO with a generic Monad in the first place?
As a general piece of advice, that might enable you to replace IO with something pure when testing.
Unit tests ought to be deterministic, which is one of the two characteristics of pure functions. Thus, being able to frame any problem in terms of pure functions makes it intrinsically testable.
In order to make a set of interactions pure and testable, you can, for example, replace m with State, and run your unit tests in the State monad. Here's an example. Here's an example with Writer.
In general, I'd recommend avoiding 'mocking' if possible. In object-oriented programming, this may be a necessary evil to enable testing, but it typically leads to hard-to-maintain code. In functional programming, unit testing is much easier, but it typically requires you to design the modules of your application in a functional style.
Introducing a type class as something equivalent to an object-oriented interface or base class is unlikely to lead to a a functional design. This will pull you towards a programming model where the (impure) interactions are at the centre of your application architecture. That's exactly what makes object-oriented programming so difficult.
In functional programming, you're much better off pushing the impure interactions to the edge of the system. This'll enable you to unit test your (pure) domain logic, while the IO remains concrete.

Related

Unit testing higher order functions in F#

Take the following F# example:
let parse mapDate mapLevel mapMessge (groups : string list) =
{
DateTime =
mapDate(
groups.[2] |> Int32.Parse,
groups.[0] |> Int32.Parse,
groups.[1] |> Int32.Parse)
Level = mapLevel groups.[3]
Message = mapMessge groups.[4]
}
I can unit test the map functions independently that's ok, but how do I unit test that this function calls the functions passed in as arguments correctly?
In C# I would use mocks and verify the calls to them. I recently watched a pluralsight video that talked about how functional languages tend to use stubs instead of mocks. Here I could pass in a function that throws if it doesn't get the expected arguments but I'm not really sold on this approach.
I was just wondering if there were any patterns in functional programming in general for unit testing higher-order functions like this?
Well, let me disagree with given answer. Actually, there is a nice way to test higher order functions without even bothering about concrete types they might take (I consider typical HOF to be totally generic, however there is no difference: approach I suggest will work with more strict HFO rightly).
Let's take something really simple, something everyone is familiar with. How about ['t] -> ['t] function? It takes a single argument - a list of whatever type and returns list of the same type. Traditional OOP approach wouldn't work here: one need's to put a restriction on 't and test somewhat specific parameters of that type; the only way to make author to feel more confident with his implementation, is to increase unit tests numbers.
There is really great stuff named "category theory" in math. It's comparatively new filed of mathematics and studies things from the outside rather from than inside. In order to be able to describe things "from the outside" you need take a thing you're interested in and force it to interact with something you already know deep enough. Thus, category theory teaches to describe things in terms of their interrelations with other things. Can't we do the same here?..
Indeed, we can. That's actually quite easy: we got a f : ['t] -> ['t] already, but is there anything else such that we could make both interact and define something common - something that holds for each and every interaction regardless of any other factors? Let's take any g: 't -> 'y. Now we able to state: g (List.head (f ...) = List.head (List.map g (f ...)). I assume a certain argument of type ['t] to substitute .... Please note: given property is universal: it would hold for any pure functions composition of specified signatures regardless of their implementation. Also note how generic yet obvious it is: there are only two distinct "objects" interacting with each other via "composition", which could also be rewritten in terms of standard F#'s (|>), (<|) operators.
Now the fact is that for any higher order (pure) function there exists such kind of universal property; mostly, there are dozens of them. Thus one able to specify their properties in terms of composition (which is regular for FP) staying at the generic level. Having such a properties in the explicit form gives one chance to autogenerate hundreds of tests, based on inputs different not only by their values (which normally done by unit tests, except the fact they are rarely autogenerated), but also by types.
Pure functions are easier because you just have to test the outputs of your parse function. You shouldn't ever need to test using side effects like you do in imperative programming.
When writing most of your unit tests, you generally use the most simple possible for your function arguments, like identity or similar. Then you'd write one test named something like "mapLevel is applied to fourth group" where instead you make mapLevel something that's easy to recognize as changed, like toUpper. This lets you make sure you didn't accidentally copy/paste mapLevel to more than one output. Then a similar test for mapMessge.

How to test an abstract data type?

Suppose we have a module that defines an abstract type T:
module AbstractType (T, lexer) where
data T = T String deriving Show
lexer = (fmap T) . words
(Note that we do not export any type constructors for T, so the user would not be able to draft an instance by hand.)
How does one unit test lexer function?
Sure we may use the Show property of T, like this:
module Main where
import AbstractType
main = test
(show $ lexer "summer is miles and miles away")
"[T \"summer\",T \"is\",T \"miles\",T \"and\",T \"miles\",T \"away\"]"
test :: (Eq a) => a -> a -> IO ()
test expression expectation
| expression == expectation = putStrLn "Test passed."
| otherwise = error "Test failed."
— But this is both not beautiful and unfit for cases when our abstract type is not an instance of a class that permits casting to another, constructable type.
Is there a remedy?
P.S. To provide some justification for the case: suppose we have a chain of functions like parser . lexer that we can integration test and see if the whole of it works. As the chain at hand gets more complex, it may nevertheless become desirable to unit test each link individually.
The example is a simplified excerpt from an actual toy text processor I am in the process of writing.
The generally accepted best practice is, for an exposed module A, to create an internal module A.Internal that is either:
Exposed but documented to be unstable or unsafe.
Not exposed to the users of the package, but only to the testing facilities. (This is made possible by the internal libraries feature released in Cabal 2.0.)
It is my understanding that functions that are not exposed enjoy more radical optimizations, particularly inlining. I am not sure if it applies to functions in internal libraries too.
On the other hand, situations often arise when a user desperately needs some internal feature of your library and ends up forking and patching it to gain access. This is, of course, unfortunate and undesirable.
I would say generally that the implementation of an abstract type is best kept in an internal library as a safety measure, but you should use your judgement in each particular case.

unit test private methods in F#

Let's say we have a class
type ThisClassIsComplicated () =
let calculateSomething a b =
a + b
In this case calculateSomething is trivial, but if it would be more complicated it may make sense to verify that the calculations done there are correct.
It might make sense to use a unit testing framework to test that private methods.
My question: how to unit test private methods in F#?
Some random thoughts:
The selected answer here, suggests to use the InternalsVisibleTo attribute which anyway is applicable only to internalmethods.
What is the route specific to F# if any? Is this better in a F# design?
let calculateSomething a b = a + b
type ThisClassIsComplicated () =
member this.Calculate a b = calculateSomething a b
Maybe the scope of calculateSomething could be even narrowed down by having a nested module.
If you feel like your code is too complicated to test it from the outside, use the latter option. And in case you want to test an inner function like
let myComplicatedOperation input =
let calculateSomething a b =
a + b
calculateSomething (fst input) (snd input)
you can always rewrite it with currying like this:
let myComplicatedOperation calculateSomething input =
calculateSomething (fst input) (snd input)
Your question does not seem to be directly related to F# though. The general way to test private methods is typically by extracting a class (or, in F#, you can also just extract a let bound function). And making your testee public on that other class / function.
I think that loosening access restrictions in a class/module to facilitate testing is often a bad idea. If you have decided something is irrelevant to know for the outside world, you wanting to test it doesn't make it any less irrelevant.
Can't you just have a public method/function in your class/module that does the testing?
type ThisClassIsComplicated () =
let calculateSomething a b =
a + b
member private this.TestInstance () =
printfn "%A" <| calculateSomething 1 2
static member Test () =
(new ThisClassIsComplicated()).TestInstance()
You can use Impromptu Interface to invoke private methods.
For example, I test the function calcNodeLabel at
https://code.google.com/p/fseye/source/browse/trunk/FsEye/Forms/WatchTreeView.fs#73 like so: https://code.google.com/p/fseye/source/browse/trunk/Test.FsEye/WatchTreeViewLabelCalculatorTests.fs#54
But you need to be careful testing hidden functions in F#: it's an implementation detail of the compiler how the function will actually be compiled (e.g. as a method, as a delegate, as a ...).
Folks will warn generally against testing private methods, but I think it is a bit simplistic to say "never test private methods", since such a declaration takes for granted that access levels as specified in the .NET framework are the only way they could be.
For example, calcNodeLabel in my example should indeed be hidden from the great wide world, but I would consider it part of the internal contract of the class. Of course, you could argue that the class view data and the view itself should be separated, but the point stands: all models are imperfect!

How do I encapsulate object constructors and destructors in haskell

I have Haskell code which needs to interface with a C library somewhat like this:
// MyObject.h
typedef struct MyObject *MyObject;
MyObject newMyObject(void);
void myObjectDoStuff(MyObject myObject);
//...
void freeMyObject(MyObject myObject);
The original FFI code wraps all of these functions as pure functions using unsafePerformIO. This has caused bugs and inconsistencies because the sequencing of the operations is undefined.
What I am looking for is a general way of dealing with objects in Haskell without resorting to doing everything in IO. What would be nice is something where I can do something like:
myPureFunction :: String -> Int
-- create object, call methods, call destructor, return results
Is there a nice way to achieve this?
The idea is to keep passing a baton from each component to force each component to be evaluated in sequence. This is basically what the state monad is (IO is really a weird state monad. Kinda).
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
import Control.Monad.State
data Baton = Baton -- Hide the constructor!
newtype CLib a = CLib {runCLib :: State Baton a} deriving Monad
And then you just string operations together. Injecting them into the CLib monad will mean they're sequenced. Essentially, you're faking your own IO, in a more unsafe way since you can escape.
Then you must ensure that you add construct and destruct to the end of all CLib chains. This is easily done by exporting a function like
clib :: CLib a -> a
clib m = runCLib $ construct >> m >> destruct
The last big hoop to jump through is to make sure that when you unsafePerformIO whatever's in construct, it actually gets evaluated.
Frankly, this is all kinda pointless since it already exists, battle proven in IO. Instead of this whole elaborate process, how about just
construct :: IO Object
destruct :: IO ()
runClib :: (Object -> IO a) -> a
runClib = unsafePerformIO $ construct >>= m >> destruct
If you don't want to use the name IO:
newtype CLib a = {runCLib :: IO a} deriving (Functor, Applicative, Monad)
My final solution. It probably has subtle bugs that I haven't considered, but it is the only solution so far which has met all of the original criteria:
Strict - all operations are sequenced correctly
Abstract - the library is exported as a stateful monad rather than a leaky set of IO operations
Safe - the user can embed this code in pure code without using unsafePerformIO and they can expect the result to be pure
Unfortunately the implementation is a bit complicated.
E.g.
// Stack.h
typedef struct Stack *Stack;
Stack newStack(void);
void pushStack(Stack, int);
int popStack(Stack);
void freeStack(Stack);
c2hs file:
{-# LANGUAGE ForeignFunctionInterface, GeneralizedNewtypeDeriving #-}
module CStack(StackEnv(), runStack, pushStack, popStack) where
import Foreign.C.Types
import Foreign.Ptr
import Foreign.ForeignPtr
import qualified Foreign.Marshal.Unsafe
import qualified Control.Monad.Reader
#include "Stack.h"
{#pointer Stack foreign newtype#}
newtype StackEnv a = StackEnv
(Control.Monad.Reader.ReaderT (Ptr Stack) IO a)
deriving (Functor, Monad)
runStack :: StackEnv a -> a
runStack (StackEnv (Control.Monad.Reader.ReaderT m))
= Foreign.Marshal.Unsafe.unsafeLocalState $ do
s <- {#call unsafe newStack#}
result <- m s
{#call unsafe freeStack#} s
return result
pushStack :: Int -> StackEnv ()
pushStack x = StackEnv . Control.Monad.Reader.ReaderT $
flip {#call unsafe pushStack as _pushStack#} (fromIntegral x)
popStack :: StackEnv Int
popStack = StackEnv . Control.Monad.Reader.ReaderT $
fmap fromIntegral . {#call unsafe popStack as _popStack#}
test program:
-- Main.hs
module Main where
import qualified CStack
main :: IO ()
main = print $ CStack.runStack x where
x :: CStack.StackEnv Int
x = pushStack 42 >> popStack
build:
$ gcc -Wall -Werror -c Stack.c
$ c2hs CStack.chs
$ ghc --make -Wall -Werror Main.hs Stack.o
$ ./Main
42
Disclaimer: I've never actually worked with C stuff from Haskell, so I am not speaking from experience here.
But what springs to mind for me is to write something like:
withMyObject :: NFData r => My -> Object -> Constructor -> Params -> (MyObject -> r) -> r
You wrap the C++ constructor/destructor as IO operations. withMyObject uses IO to sequence the constructor, calling the user-specified function, calling the destructor, and returning the result. It can then unsafePerformIO that entire do block (as opposed to the individual operations within it, which you've already cooking doesn't work). You need to use deepSeq too (which is why the NFData constraint is there), or laziness could defer the use of the MyObject until after it's been destructed.
The advantages of this is are:
You can write pure MyObject -> r functions using whatever ordinary code you like, no monads required
You can decide to construct a MyObject in order to call such functions in the middle of other ordinary pure code, with the help of withMyObject
You can't forget to call the destructor when you use withMyObject
You can't use the MyObject after calling the destructor on it1
There is only one (small) place in your system where you use unsafePerformIO, and therefore that's the only place you have to carefully worry about whether you've got the sequencing correct to justify that it's safe after all. There's also only one place you have to worry about making sure you use the destructor properly.
It's basically the "construct, use, destruct" pattern with the particulars of the "use" step abstracted out as a parameter so that you can has a single implementation cover every time you need to use that pattern.
The main disadvantage is that it's a bit awkward to construct a MyObject and then pass it to several unrelated functions. You have to bundle them up into a function that returns a tuple of each of the original results, and then use withMyObject on that. Alternatively if you also expose the IO versions of he constructor and destructor separately the user has the option of using those if IO is less awkward than making wrapper functions to pass to withMyObject (but then it's possible for the user to accidentally use the MyObject after freeing it, or forget to free it).
1 Unless you do something silly like use id as the MyObject -> r function. Presumably there's no NFData MyObject instance though. Also that sort of error would tend to come from willful abuse rather than accidental misunderstanding.

Clojure Style Agents in F#

I'm trying to code some Clojure style Agents in F# using MailboxProcessors. Here's what I have so far:
namespace GameEngine
type Agent<'T>(inital:'T) =
let mutable state:'T = inital
let queue = new MailboxProcessor<'T -> 'T>( fun inbox ->
let rec loop count =
async {
let! msg = inbox.Receive()
state <- msg(state)
return! loop(count + 1)
}
loop 0)
do
queue.Start()
member self.Send(action:'T -> 'T) =
queue.Post(action)
member self.Deref() =
state
So the basic idea is that we have a mutable state that can be updated by calling .Send(). My question is, will my messages ever be out of order? If msg A is sent before B will the async function above always process A before B?
Seems like there should be a class like this already in F#? Am I reinventing the wheel?
If msg A is sent before B will the async function above always process A before B?
Yes. (You can see the code for Mailbox
http://fsharppowerpack.codeplex.com/SourceControl/changeset/view/54799#970072
browse to compiler\2.0\Nov2010\src\fsharp\FSharp.Core\control.fs, and eventually see e.g.
member x.Post(msg) =
lock syncRoot (fun () ->
arrivals.Enqueue(msg);
...
which shows it's just a queue under a lock.)
Seems like there should be a class like this already in F#? Am I reinventing the wheel?
Well, it's not immediately clear to me how this is different from just updating a mutable global variable willy-nilly (modulo atomicity of simultaneous updates; you said "before" in the question, so I am unclear if that aspect matters to you). What's the context for wanting this?
There is no built-in implementation of the Clojure-style Agent.
I also at one point worked up a quick and dirty F# implementation similar to yours, but did not take the time to consider all the correctness issues involved; in particular, is it not true that 'T may be a value type (a struct) larger than 64 bits (or 32 bits as the case may be) which could cause a tear (I presume that Clojure like Java doesn't have structs to worry about here). Perhaps an F# generic type constraint ('T when 'T : not struct) would be needed?