Groovy: test-oriented output for junit - unit-testing

Question

I'm trying to follow the Groovy Testing Guide to write some JUnit tests for my Groovy code, but I can't get it to output anything useful. For example, if I have this:
class Test extends GroovyTestCase {
void testCase1() {
assertTrue true
assertEquals 1, 1
}
}
Then I get this:
$ groovy Test.groovy
.
Time: 0.045
OK (1 test)
This is fine as a basic output, but it would be nice if I could get some info about what test cases were actually run. A bigger issue though, is the output when something fails:
class Test extends GroovyTestCase {
void testCase1() {
assertTrue true
assertEquals 1, 1
}
void testCase2() {
assertEquals 1, 2
}
void testCase3() {
assertEquals 1, 2
}
}
Results in:
$ groovy Test.groovy
..F.F
Time: 0.046
There were 2 failures:
1) testCase2(Test)junit.framework.AssertionFailedError: expected:<1> but was:<2>
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1467)
at org.codehaus.groovy.runtime.callsite.StaticMetaClassSite.callStatic(StaticMetaClassSite.java:65)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:214)
at Test.testCase2(Test.groovy:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1446)
at org.codehaus.groovy.runtime.InvokerHelper.invokeStaticMethod(InvokerHelper.java:951)
at org.codehaus.groovy.runtime.InvokerHelper.invokeStaticMethod(InvokerHelper.java:83)
at groovy.lang.GroovyShell.runJUnit3Test(GroovyShell.java:375)
at groovy.lang.GroovyShell.runScriptOrMainOrTestOrRunnable(GroovyShell.java:295)
at groovy.lang.GroovyShell.run(GroovyShell.java:518)
at groovy.lang.GroovyShell.run(GroovyShell.java:507)
at groovy.ui.GroovyMain.processOnce(GroovyMain.java:653)
at groovy.ui.GroovyMain.run(GroovyMain.java:384)
at groovy.ui.GroovyMain.process(GroovyMain.java:370)
at groovy.ui.GroovyMain.processArgs(GroovyMain.java:129)
at groovy.ui.GroovyMain.main(GroovyMain.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.codehaus.groovy.tools.GroovyStarter.rootLoader(GroovyStarter.java:109)
at org.codehaus.groovy.tools.GroovyStarter.main(GroovyStarter.java:131)
2) testCase3(Test)junit.framework.AssertionFailedError: expected:<1> but was:<2>
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1467)
at org.codehaus.groovy.runtime.callsite.StaticMetaClassSite.callStatic(StaticMetaClassSite.java:65)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:214)
at Test.testCase3(Test.groovy:14)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1446)
at org.codehaus.groovy.runtime.InvokerHelper.invokeStaticMethod(InvokerHelper.java:951)
at org.codehaus.groovy.runtime.InvokerHelper.invokeStaticMethod(InvokerHelper.java:83)
at groovy.lang.GroovyShell.runJUnit3Test(GroovyShell.java:375)
at groovy.lang.GroovyShell.runScriptOrMainOrTestOrRunnable(GroovyShell.java:295)
at groovy.lang.GroovyShell.run(GroovyShell.java:518)
at groovy.lang.GroovyShell.run(GroovyShell.java:507)
at groovy.ui.GroovyMain.processOnce(GroovyMain.java:653)
at groovy.ui.GroovyMain.run(GroovyMain.java:384)
at groovy.ui.GroovyMain.process(GroovyMain.java:370)
at groovy.ui.GroovyMain.processArgs(GroovyMain.java:129)
at groovy.ui.GroovyMain.main(GroovyMain.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.codehaus.groovy.tools.GroovyStarter.rootLoader(GroovyStarter.java:109)
at org.codehaus.groovy.tools.GroovyStarter.main(GroovyStarter.java:131)
FAILURES!!!
Tests run: 3, Failures: 2, Errors: 0
This isn't great output. I don't need a massive stack trace like this, especially one that doesn't actually tell me anything about the code under test. It would be much better if I could get something more test-oriented, like:
testCase1: PASS
testCase2: FAIL
junit.framework.AssertionFailedError: expected:<1> but was:<2>
testCase3: FAIL
junit.framework.AssertionFailedError: expected:<1> but was:<2>
FAILURES!!!
Tests run: 3, Failures: 2, Errors: 0
Is there a way to get more test-oriented output from JUnit in Groovy?

Answer 1

I believe, you could just use in built method assert with a user message.
void testCase2() {
assert 1 == 2, 'expected differs from actual'
}

Answer 2

If you want to try spock, here's the code you need:
#Grab('org.spockframework:spock-core:1.1-groovy-2.4')
import spock.lang.Specification
class Test extends Specification {
def "test case 1"() {
expect:
1 == 1
}
def "test case 2"() {
expect:
1 == 2
}
def "test case 3"() {
expect:
1 == 2
}
}
I also would recommend (depending on your use-case -- which you don't specify) using a build tool such as maven or gradle.

Answer 3

I THINK they problem is that you are running the unit test with Java, this is exactly how asserts work, they throw an exception.
Try running org.junit.runner.JUnitCore and having it run your test, this should know how to summarize the results and print them (in various formats, even XML) without the exception.
By the way, the XML output works great with groovy's XML parsing abilities, also groovy's assert is good but expects a fixed-width font for drawing it's little picture--if you are just viewing the results yourself this is great, but if you are working with them pragmatically, junit's Assert class might leave better messages.

Answer 4

It's a good idea to have the stacktrace, because you'll want it if the class you're testing actually threw an exception, as opposed to just generating an AssertionFailedError.
But its often the case with groovy exceptions that you only want to know about your own methods, not all the underlying groovy stuff, so you use StackTraceUtils.sanitize:
import org.codehaus.groovy.runtime.StackTraceUtils
class Test extends GroovyTestCase {
void testCase1() {
assertTrue true
assertEquals 1, 1
}
void testCase2() {
assertEquals 1, 1
}
void testCase3() {
try {
assertEquals 1, 2
} catch (Throwable e) {
throw StackTraceUtils.sanitize(e)
}
}
}
This gives:
...F
Time: 0.047
There was 1 failure:
1) testCase3(Test)junit.framework.AssertionFailedError: expected:<1> but was:<2>
at Test.testCase3(Test.groovy:15)
at org.apache.groovy.plugin.DefaultRunners$Junit3TestRunner.run(DefaultRunners.java:99)
FAILURES!!!
Tests run: 3, Failures: 1, Errors: 0
Maybe somebody has a way that you can avoid writing this in every single test case...