How do I make the `groovy` command exit non-zero for failing JUnit4 tests? - unit-testing

When running JUnit tests with the groovy command using the built-in automatic test runner, it exits 0 even when tests fail. I'd like the command to exit non-zero if there are test failures. Is there a way I can do this?
#!/usr/bin/env groovy
import org.junit.*
class BasicTest {
#Test
void test_failure() {
assert false
}
}
$ groovy --version
Groovy Version: 3.0.2 JVM: 13.0.2 Vendor: Oracle Corporation OS: Mac OS X
$ groovy basic_test.groovy
JUnit 4 Runner, Tests: 1, Failures: 1, Time: 8
Test Failure: test_failure(BasicTest)
Assertion failed:
assert false
at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:434)
...
$ echo $?
0
Thanks folks!

Not seen people running tests like that before... (writing them as a *nix script)
You can do this though, to catch a failure, and exiting with 1
#!/usr/bin/env groovy
import org.junit.*
import org.junit.rules.*
class BasicTest {
#Rule
public TestRule watchman = [
failed: {e, d ->
println d
e.printStackTrace()
System.exit(1)
}
] as TestWatcher
#Test
void test_failure() {
assert false
}
}

Related

Regex: Negative matcher on javascript string ^(?!\\[functional\\]).+$ files fails to exclude [functional]

In package.json I have 2 script commands:
"test:unit": "jest --watch --testNamePattern='^(?!\\[functional\\]).+$'",
"test:functional": "jest --watch --testNamePattern='\\[functional\\]'",
copying ^(?!\\[functional\\]).+$ into https://regex101.com/, it does not match the test string below inside argument 1 of describe()
describe("[functional] live tests", () => {
When changed to ([functional]).+$, the pattern does match. I have to remove a pair of \ on each end to remove escapes for .json files (I think).
Here is what I see when running npm run test:unit in my project root:
// the functional test runs (not desired)
$ npm run test:unit
functions/src/classes/__tests__/Functional.test.ts:30:47 - error TS2339: Property 'submit' does not exist on type 'Element'.
30 await emailForm.evaluate(form => form.submit());
~~~~~~
RUNS ...s/__tests__/Functional.test.ts
Test Suites: 1 failed, 1 skipped, 3 passed, 4 of 5 total
Tests: 2 skipped, 16 passed, 18 total
Snapshots: 0 total
Time: 8.965s, estimated 27s
Ran all test suites with tests matching "^(?!\[functional\]).+$".
Active Filters: test name /^(?!\[functional\]).+$/
The functional tests are not built out which explains the syntax error, it's not important here. The key issue, is why the tests were not skipped.
I believe the problem has to do with the regex negative matcher. The positive matcher without the ! only matches tests that have, or are nested in a describe block with [functional]
$ npm run test:functional
Test Suites: 1 failed, 4 skipped, 1 of 5 total
Active Filters: test name /\[functional\]/
Anyone know why the negative regex pattern is failing during npm run test:unit ?
Instead of a regex fix I changed the flag on the unit testing script to an ignore, then copying the matching pattern for [functional]:
"test:unit": "jest --watch --testIgnorePattern='\\[functional\\]'",
"test:functional": "jest --watch --testNamePattern='\\[functional\\]'",

Unit testing in vuejs

I am trying to configure/run my first unit test for Vuejs. But I can't get past the configuration issues. I have tried installing the libraries but for some reason I keep getting errors.
Here is what an example of my code looks like:
My directory structure:
hello/
dist/
node_modules/
src/
components/
hello.vue
test/
setup.js
test.spec.js
.babelrc
package.json
webpack.config.js
Contents inside my files
src/components/hello.vue
<template> <div> {{message}} </div> </template>
<script>
export default {
name: 'hello',
data () { return message: 'Hi' },
created () {
// ...
}
}
test/setup.js
// setup JSDOM
require('jsdom-global')()
// make expect available globally
global.expect = require('expect')
test/test.spect.js
import { shallow } from 'vue/test-utils'
import { hello} from '../../../src/components/hello.vue'
describe('hello', () => {
// just testing simple data to see if it works
expect(1).toBe(1)
})
.babelrc
{
"env": {
"development": {
"presets": [
[
"env",
{
"modules": false
}
]
]
},
"test": {
"presets": [
[
"env",
{
"modules": false,
"targets": {
"node": "current"
}
}
]
],
"plugins": [
"istanbul"
]
}
}
}
package.json
...
"scripts": {
"build": "webpack -p",
"test": "cross-env NODE_ENV=test nyc mocha-webpack --webpack-config webpack.config.js --require test/setup.js test/**/*.spec.js"
},
"devDependencies": {
"babel-core": "^6.26.0",
"babel-loader": "^7.1.2",
"babel-preset-env": "^1.6.1",
"cross-env": "^5.1.1",
"css-loader": "^0.28.7",
"file-loader": "^1.1.5",
"node-sass": "^4.7.2",
"sass-loader": "^6.0.6",
"vue-loader": "^13.5.0",
"vue-template-compiler": "^2.5.9",
"webpack": "^3.10.0",
"webpack-dev-server": "^2.9.7",
"jsdom": "^11.3.0",
"jsdom-global": "^3.0.2",
"mocha": "^3.5.3",
"mocha-webpack": "^1.0.0-rc.1",
"nyc": "^11.4.1",
"expect": "^21.2.1",
"#vue/test-utils": "^1.0.0-beta.12"
},
...
"nyc": {
"include": [
"src/**/*.(js|vue)"
],
"instrument": false,
"sourceMap": false
}
and finally my webpack.config.js
...
if(process.env.NODE_ENV == "test") {
module.exports.externals = [ require ('webpack-node-externals')()]
module.exports.devtool = 'inline-cheap-module-source-map'
}
now when I run npm test from my root folder hello/ I get this error:
> hello#1.0.0 test C:\Users\john\vue-learn\hello
> npm run e2e
> hello#1.0.0 e2e C:\Users\john\vue-learn\hello
> node test/e2e/runner.js
Starting selenium server... started - PID: 12212
[Test] Test Suite
=====================
Running: default e2e tests
× Timed out while waiting for element <#app> to be present for 5000 milliseconds. - expected "visible" but got: "not found"
at Object.defaultE2eTests [as default e2e tests] (C:/Users/john/Google Drive/lab/hello/test/e2e/specs/test.js:13:8)
at _combinedTickCallback (internal/process/next_tick.js:131:7)
FAILED: 1 assertions failed (20.281s)
_________________________________________________
TEST FAILURE: 1 assertions failed, 0 passed. (20.456s)
× test
- default e2e tests (20.281s)
Timed out while waiting for element <#app> to be present for 5000 milliseconds. - expected "visible" but got: "not found"
at Object.defaultE2eTests [as default e2e tests] (C:/Users/john/Google Drive/lab/hello/test/e2e/specs/test.js:13:8)
at _combinedTickCallback (internal/process/next_tick.js:131:7)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! hello#1.0.0 e2e: `node test/e2e/runner.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the hello#1.0.0 e2e script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\john\AppData\Roaming\npm-cache\_logs\2018-04-03T23_53_15_976Z-debug.log
npm ERR! Test failed. See above for more details.
I don't know why this happens. When I installed my webpack project at first I didn't install a testing library with the npm init command so there are no conflicts, but still I get that error:
update (after bounty)
I'm just trying to test my vuejs application. Hopefully with jasmine/karma. If anyone knows how to integrate these into a simple app and run the firsts test, I can take it from there. My problem is not writing tests but configuring it
So first thing you didn't need to enable the end to end testing in your project. I would say start fresh
$ npm install -g vue-cli
$ vue init webpack vue-testing
? Project name vue-testing
? Project description A Vue.js project
? Author Tarun Lalwani <tarun.lalwani#payu.in>
? Vue build standalone
? Install vue-router? Yes
? Use ESLint to lint your code? Yes
? Pick an ESLint preset Standard
? Set up unit tests Yes
? Pick a test runner karma
? Setup e2e tests with Nightwatch? No
? Should we run `npm install` for you after the project has been created? (recommended) yarn
Say N to Setup e2e tests with Nightwatch and use Karma for the Pick a test runner.
$ npm test
> vue-testing#1.0.0 test /Users/tarun.lalwani/Desktop/tarunlalwani.com/tarunlalwani/workshop/ub16/so/vue-testing
> npm run unit
> vue-testing#1.0.0 unit /Users/tarun.lalwani/Desktop/tarunlalwani.com/tarunlalwani/workshop/ub16/so/vue-testing
> cross-env BABEL_ENV=test karma start test/unit/karma.conf.js --single-run
07 04 2018 21:35:28.620:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
07 04 2018 21:35:28.629:INFO [launcher]: Launching browser PhantomJS with unlimited concurrency
07 04 2018 21:35:28.645:INFO [launcher]: Starting browser PhantomJS
07 04 2018 21:35:32.891:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: Connected on socket M1HeZIiOis3eE3mLAAAA with id 44927405
HelloWorld.vue
✓ should render correct contents
PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 1 of 1 SUCCESS (0.061 secs / 0.041 secs)
TOTAL: 1 SUCCESS
=============================== Coverage summary ===============================
Statements : 100% ( 2/2 )
Branches : 100% ( 0/0 )
Functions : 100% ( 0/0 )
Lines : 100% ( 2/2 )
================================================================================
Now your npm test would work fine.
According to the error logs you provide here, the failing tests that you spot are the End to End ones. Indeed, by executing the command npm test e2e you're testing using Nightwatch. See under /tests/e2e/specs. Here you should have a default test file checking that your Vue application properly create a DOM element identified as app.
The test should be the following:
// For authoring Nightwatch tests, see
// http://nightwatchjs.org/guide#usage
module.exports = {
'default e2e tests': function (browser) {
// automatically uses dev Server port from /config.index.js
// default: http://localhost:8080
// see nightwatch.conf.js
const devServer = browser.globals.devServerURL
browser
.url(devServer)
.waitForElementVisible('#app', 5000)
.assert.elementPresent('.hello')
.assert.containsText('h1', 'Welcome to Your Vue.js App')
.assert.elementCount('img', 1)
.end()
}
}
In your case this test is failing because you have probably removed the file named App.vue that is generated through vue-cli scaffolding. The error you get is because the above test checks, with a 5 seconds timeout, if a DOM node named "app" is rendered (i.e.: .waitForElementVisible('#app', 5000)).
Basically it is failing because you actually do not provide this div in your application anymore (due of App.vue removal, maybe).
So you have two options here:
restoring the App.vue file (i.e.: create a div identified as 'app' where you mount a Vue instance);
editing the end to end according to your needs.
Hope this helps!

What is the correct Spock syntax for Grails?

I have a Grails 2.5.0 app running and this test:
package moduleextractor
import grails.test.mixin.TestFor
import spock.lang.Specification
/**
* See the API for {#link grails.test.mixin.web.ControllerUnitTestMixin} for usage instructions
*/
#TestFor(ExtractorController)
class ExtractorControllerSpec extends Specification {
def moduleDataService
def mockFile
def setup() {
moduleDataService = Mock(ModuleDataService)
mockFile = Mock(File)
}
def cleanup() {
}
void "calls the moduleDataService"() {
given: 'a term is passed'
params.termCode = termCode
when: 'the getModuleData action is called'
controller.getModuleData()
then: 'the service is called 1 time'
1 * moduleDataService.getDataFile(termCode, 'json') >> mockFile
where:
termCode = "201415"
}
}
If I run grails test-app unit:spock I get this:
| Tests PASSED - view reports in /home/foo/Projects/moduleExtractor/target/test-reports
I don't understand why it sees 2 tests. I have not included spock in my BuildConfig file as it is already included in Grails 2.5.0. Also the test is not supposed to pass, as I do not have a service yet. Why does it pass?
Also when I run this grails test-app ExtractorController I get another result:
| Running 2 unit tests...
| Running 2 unit tests... 1 of 2
| Failure: calls the moduleDataService(moduleextractor.ExtractorControllerSpec)
| Too few invocations for:
1 * moduleDataService.getDataFile(termCode, 'json') >> mockFile (0 invocations)
Unmatched invocations (ordered by similarity):
None
at org.spockframework.mock.runtime.InteractionScope.verifyInteractions(InteractionScope.java:78)
at org.spockframework.mock.runtime.MockController.leaveScope(MockController.java:76)
at moduleextractor.ExtractorControllerSpec.calls the moduleDataService(ExtractorControllerSpec.groovy:27)
| Completed 1 unit test, 1 failed in 0m 3s
| Tests FAILED - view reports in /home/foo/Projects/moduleExtractor/target/test-reports
| Error Forked Grails VM exited with error
If I run grails test-app unit: I get:
| Running 4 unit tests...
| Running 4 unit tests... 1 of 4
| Failure: calls the moduleDataService(moduleextractor.ExtractorControllerSpec)
| Too few invocations for:
1 * moduleDataService.getDataFile(termCode, 'json') >> mockFile (0 invocations)
Unmatched invocations (ordered by similarity):
None
at org.spockframework.mock.runtime.InteractionScope.verifyInteractions(InteractionScope.java:78)
at org.spockframework.mock.runtime.MockController.leaveScope(MockController.java:76)
at moduleextractor.ExtractorControllerSpec.calls the moduleDataService(ExtractorControllerSpec.groovy:27)
| Completed 1 unit test, 1 failed in 0m 3s
| Tests FAILED - view reports in /home/foo/Projects/moduleExtractor/target/test-reports
| Error Forked Grails VM exited with error
First of all could somebody tell me what is the correct syntax to run spock tests?
Also what is the difference between having unit and unit: and unit:spock in the command?
(Since Spock comes with Grails 2.5.0, it will run spocks tests anyway.)
What is the correct syntax and why does it sees 2 tests instead of 1 ?
Don't be concerned with the number of tests. It's never been a problem for me. You can always check the report HTML file to see exactly what ran.
I always run my tests with either
grails test-app
or
grails test-app ExtractorController
The error you're getting means you coded the test to expect moduleDataService.getDataFile() to get called with parameters null and 'json' when controller.getModuleData() is called. However, moduleDataService.getDataFile() never got called, so the test failed.
Spock takes some getting used to. I recommend looking at examples in the Grails documentation and reading the Spock Framework Reference.
First question: for the 'grails test-app unit:spock', have you looked at the results to see the tests it says passed? The test count at the CLI can be wrong, check your results to see what actually ran (if no tests actually ran, then there were no failures).
Your test method doesn't start with 'test', nor does it have a #Test annotation, so the 'void "calls the moduleDataService"' isn't being seen as a spock test case (I believe that is the reason).
When you run 'grails test-app ExtractorController', you aren't specifying that it has to be a spock test, so grails testing finds and executes the 'calls the moduleDataService' test method.
Since spock is the de facto testing framework, you can just use:
grails test-app -unit
Second question:
#TestFor creates your controller, but if you're running a unit test, then the usual grails magic isn't happening. Your controller code is executing in isolation. If your ExtractorController usually has the moduleDataService injected, you'll have to take care of that.
I work in grails 2.4.3, and here would be my interpretation of your test (assuredly in need of tweaking since I'm inferring a lot in this example):
import grails.test.mixin.TestFor
import grails.test.mixin.Mock
import spock.lang.specification
import some.pkg.ModuleDataService // if necessary
import some.pkg.File // if necessary
#TestFor(ExtractorController)
#Mock([ModuleDataService, File])
class ExtractorControllerSpec extends Specification
def "test callsModuleDataService once for a termCode"() {
setup:
def mockFile = mockFor(File)
def mockService = mockFor(ModuleDataService, true) // loose mock
// in this mockService, we expect getDataFile to be called
// just once, with two parameters, and it'll return a mocked
// file
mockService.demand.getDataFile(1) { String termCode, String fmt ->
return mockFile.createMock()
}
controller.moduleDataService = mockService.createMock()
when:
controller.params.termCode = "201415"
controller.getModuleData()
then:
response.status == 200 // all good?
}
}
Last question: is that a Banner term code? (just curious)

Grails Unit Test Exception java.lang.Exception: No tests found matching grails test target pattern filter

I am just starting to learn Grails testing and I tried to write my first grails test.For this, I created a fresh grails project and created a controller named com.rahulserver.SomeController:
package com.rahulserver
class SomeController {
def index() { }
def someAction(){
}
}
When I created this controller, grails automatically created a com.rahulserver.SomeControllerSpec under test/unit folder.
Here is my SomeControllerSpec.groovy:
package com.rahulserver
import grails.test.mixin.TestFor
import spock.lang.Specification
/**
* See the API for {#link grails.test.mixin.web.ControllerUnitTestMixin} for usage instructions
*/
#TestFor(SomeController)
class SomeControllerSpec extends Specification {
def setup() {
}
def cleanup() {
}
void testSomeAction() {
assert 1==1
}
}
When I right click this class, and run this test, I get following:
Testing started at 5:21 PM ...
|Loading Grails 2.4.3
|Configuring classpath
.
|Environment set to test
....................................
|Running without daemon...
..........................................
|Compiling 1 source files
.
|Running 1 unit test...|Running 1 unit test... 1 of 1
--Output from initializationError--
Failure: |
initializationError(org.junit.runner.manipulation.Filter)
|
java.lang.Exception: No tests found matching grails test target pattern filter from org.junit.runner.Request$1#1f0f9da5
at org.junit.internal.requests.FilterRequest.getRunner(FilterRequest.java:35)
at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
No tests found matching grails test target pattern filter from org.junit.runner.Request$1#1f0f9da5
java.lang.Exception: No tests found matching grails test target pattern filter from org.junit.runner.Request$1#1f0f9da5
at org.junit.internal.requests.FilterRequest.getRunner(FilterRequest.java:35)
at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
|Completed 1 unit test, 1 failed in 0m 0s
.Tests FAILED
|
- view reports in D:\115Labs\grailsunittestdemo\target\test-reports
Error |
Forked Grails VM exited with error
Process finished with exit code 1
So why is it failing?
EDIT
I am using grails 2.4.3
The unit tests are defined with Spock by default:
void testSomeAction() {
assert 1==1
}
Should be written as:
void "Test some action"() {
expect:
1==1
}
See http://spockframework.github.io/spock/docs/1.0/index.html

Grails unit test case showing Completed 0 spock test, 0 failed

I am new to Grails. I am using version 2.2.2
My test cases are not running even it says test cases passed.
I get the following message after running the test case.
Resolving [test] dependencies...
Resolving [runtime] dependencies...
| Compiling 1 source files.
| Error log4j:ERROR Property missing when configuring log4j: grails
| Error log4j:ERROR Property missing when configuring log4j: grails
| Error log4j:ERROR WARNING: Exception occured configuring log4j logging: null
| Completed 0 spock test, 0 failed in 1409ms
| Tests PASSED - view reports in D:\workspace_idea\optapp\target\test-reports
#TestFor(KpiLog)
#TestMixin(DomainClassUnitTestMixin)
#Mock(KpiLog)
class KpiLogSpec extends Specification {
void "savelog"() {
prinln "*********"
when:
def kpiLog = new KpiLog(scenarioId: 1, kpiId: 2, deltaKpi: 5)
kpiLog.save(flush: true)
then:
KpiLog.list()!= null
}
void testSaveFacebookUser(){
//given
def kpiLog = new KpiLog(scenarioId: 1, kpiId: 2, deltaKpi: 5)
//adminRole.addToPermissions("*:*")
kpiLog.save()
}
}
Can some one please tell me what is it that I am doing wrong?
I am running the test case as grails test-app -unit KpiLogSpec
Here is the log4j section from the Config.groovy file
log4j = {
// Example of changing the log pattern for the default console
// appender:
//
//appenders {
// console name:'stdout', layout:pattern(conversionPattern: '%c{2} %m%n')
//}
debug 'grails.app'
error 'org.codehaus.groovy.grails.web.servlet', // controllers
'org.codehaus.groovy.grails.web.pages', // GSP
'org.codehaus.groovy.grails.web.sitemesh', // layouts
'org.codehaus.groovy.grails.web.mapping.filter', // URL mapping
'org.codehaus.groovy.grails.web.mapping', // URL mapping
'org.codehaus.groovy.grails.commons', // core / classloading
'org.codehaus.groovy.grails.plugins', // plugins
'org.codehaus.groovy.grails.orm.hibernate', // hibernate integration
'org.springframework',
'org.hibernate',
'net.sf.ehcache.hibernate'
appenders {
console name:'S', layout:pattern(conversionPattern: '%d %-5p %c - %m%n')
//rollingFile name: 'R', file:'/usr/local/jd/logs/optimizer.log',maxFileSize: '5000000KB'
rollingFile name: 'R', file:grails.config.logPath, maxFileSize: '5000000KB'
environments {
production {
appender new AWSSNSAppender(
name:'SNS' ,
topicName:config.optimizer.snsAppender.topicName,
topicSubject:config.optimizer.snsAppender.topicSubject,
awsAccessKey:config.optimizer.aws.accessKey,
awsSecretKey:config.optimizer.aws.secretKey,
threshold:Level.toLevel(optimizer.snsAppender.threshold, Level.ERROR)
)
}
}
}
info R: ['NotifierService', 'aggDataStackLog','constraintGroupLog','pageFilterLog','alertDebugLog','dacCacLog','tpFlagExportImportLog','timelog','calculationProgress','dictionaryLog','loginServiceLog', 'calculateScenarioLog','connectionLog','dataLabelServiceLog','cross-section-service','qe-basic-executor-service','qe-plan-enumerator-impl-service','qe-basic-planenum-ssservice','ct-dimension-hierarchy-service','cbRuleLog'],additivity:true
error SNS: ['aggDataStackLog','constraintGroupLog','pageFilterLog','alertDebugLog','dacCacLog','tpFlagExportImportLog','timelog','calculationProgress','dictionaryLog','loginServiceLog', 'calculateScenarioLog','connectionLog','dataLabelServiceLog','cross-section-service','qe-basic-executor-service','qe-plan-enumerator-impl-service','qe-basic-planenum-ssservice','ct-dimension-hierarchy-service','cbRuleLog'],additivity:true
//info SNS: ['aggDataStackLog','calculateScenarioLog']
/*root {
error 'R'
additivity = true
} */
}
Here is the test code which I ran.
#TestMixin(GrailsUnitTestMixin)
class FooSpec extends Specification {
def setup() {
}
def cleanup() {
}
void "test something"() {
println "****************testing in real "
assertTrue(1==1)
}
}
Run the test case like this
grails test-app unit: KpiLog
The important thing is that you use unit: instead of -unit and KpiLog instead of KpiLogSpec
Then do not define the variable log.
def log = new KpiLog(scenarioId: 1, kpiId: 2, deltaKpi: 5)
It is reserved for logging in Grails classes (controllers, services, ...). Rename the variable from log to kpiLog
def kpiLog = new KpiLog(scenarioId: 1, kpiId: 2, deltaKpi: 5)
A simple log configuration can be
log4j = {
appenders {
console name: 'stdout', layout: pattern(conversionPattern: '%d [%t] %-5p %c - %m%n')
}
error 'org.codehaus.groovy.grails.web.servlet', // controllers
'org.codehaus.groovy.grails.web.pages', // GSP
'org.codehaus.groovy.grails.web.sitemesh', // layouts
'org.codehaus.groovy.grails.web.mapping.filter', // URL mapping
'org.codehaus.groovy.grails.web.mapping', // URL mapping
'org.codehaus.groovy.grails.commons', // core / classloading
'org.codehaus.groovy.grails.plugins', // plugins
'org.codehaus.groovy.grails.orm.hibernate', // hibernate integration
'org.springframework',
'org.hibernate',
'net.sf.ehcache.hibernate'
}
Finally, it is working. It was because of the jar msutil in the lib folder. I got this information from one of my friends that it has some incompatibility. I changed the jar which he sent and things started working.
Thanks a lot saw303 for keeping patience with my questions.