How to Run Multiple Test Files with Haskell Stack Project - unit-testing

Question

I want to setup an existing Haskell project with Stack. The existing project uses multiple files under a test directory; these separate test files by default, Stack (or cabal?) appears to utilize a single test/Spec.hs for testing. How can I continue to use multiple files with this project?
NOTE: I'm learning Haskell, and this project approaches my learning from a "kata" approach. So tests are isolate to focus on one aspect of the language at a time.

Answer 1

Here is a setup for a directory structure like this
> tree
.
├── example.cabal
├── app
│   └── Main.hs
├── ChangeLog.md
├── LICENSE
├── Setup.hs
├── src
│   ├── A
│   │   └── C.hs
│   ├── A.hs
│   └── B.hs
├── stack.yaml
└── tst
├── integration
│   └── Spec.hs
└── unit
├── A
│   └── CSpec.hs
├── ASpec.hs
├── BSpec.hs
└── Spec.hs
you want to have integration tests that are separate from the usual unit tests and several sub-modules that correspond to each module in your src-folder
first of all you need to add the test suites to your
example.cabal file
name: example
...
-- copyright:
-- category:
build-type: Simple
extra-source-files: ChangeLog.md
cabal-version: >=1.10
executable testmain
main-is: Main.hs
hs-source-dirs: app
build-depends: base
, example
library
exposed-modules: A.C,A,B
-- other-modules:
-- other-extensions:
build-depends: base >=4.9 && <4.10
hs-source-dirs: src
default-language: Haskell2010
test-suite unit-tests
type: exitcode-stdio-1.0
main-is: Spec.hs
hs-source-dirs: tst/unit
build-depends: base
, example
, hspec
, hspec-discover
, ...
test-suite integration-tests
type: exitcode-stdio-1.0
main-is: Spec.hs
hs-source-dirs: tst/integration
build-depends: base
, example
, hspec
, ...
put the following in your tst/unit/Spec.hs it is from hspec-discover and it discovers (hence the name) all modules of the form ...Spec.hs and executes the spec function from each of those modules.
tst/unit/Spec.hs
{-# OPTIONS_GHC -F -pgmF hspec-discover #-}
just this single line
Other test files
then add your unit tests in your ASpec.hs, and others in BSpec.hs,CSpec.hs and your Spec.hs in the tst/integration folder
module ASpec where
import Test.Hspec
import A
spec :: Spec
spec = do
describe "Prelude.head" $ do
it "returns the first element of a list" $ do
head [23 ..] `shouldBe` (23 :: Int)
it "returns the first element of an *arbitrary* list" $
property $ \x xs -> head (x:xs) == (x :: Int)
it "throws an exception if used with an empty list" $ do
evaluate (head []) `shouldThrow` anyException
you can then compile and run your tests with
$> stack test
# now all your tests are executed
$> stack test :unit-tests
# now only the unit tests run
$> stack test :integration-tests
# now only the integration tests run
Sources
You can find all the examples at https://hspec.github.io, if you want to know more about hspec-style testing I guess it would be best to start there. For the stack - go to https://haskellstack.org - there is some information about testing/benchmarking there - I mean about running tests and benchmarks.
For different testing style in haskell see HUnit, QuickCheck, Smallcheck, doctests (If I forgot one, my dearest apologies - those are the ones that I use regularly as well).

Answer 2

Here is a solution with just stack and HUnit. Nothing against hspec, htf, tasty, etc etc, but likewise there is not much glue needed even without those, if you're already using HUnit. It doesn't require editing the cabal file. The original question implies use of hspec, so the #epsilonhalbe is still closer on that criteria.
.
├─stack.yaml
├─package.yaml
├─src/
| ├─A.hs
| ├─B.hs
| ├─Main.hs
├─tst/
| ├─ATest.hs
| ├─BTest.hs
| ├─Main.hs
Example package.yaml file:
name: example
version: 0.1.0.0
dependencies:
- HUnit >= 1.6.1.0 && < 2
library:
source-dirs: src
executables:
example-exe:
main: Main.hs
source-dirs: src
dependencies:
- example
tests:
example-test:
main: Main.hs
source-dirs: tst
dependencies:
- example
- HUnit
In ATest.hs and BTest.hs declare list of tests called huTests in the usual HUnit way, eg,
huTests = ["egTest" ~: "a" ~=? "a"].
Then tst/Main.hs has the glue in a common HUnit idiom (see eg this answer):
import ATest (huTests)
import BTest (huTests)
import System.Exit
import Test.HUnit
main :: IO ()
main = do
results <- runTestTT $
test (ATest.huTests ++ BTest.huTests)
if errors results + failures results == 0 then
putStrLn "Tests passed."
else
die "Tests failed."