I am using gulp and mocha to run unit tests which is part of a gulp workflow to run my reactjs app. The unit test works:
gulp.task('mocha', function () {
return gulp
.src(['test/*.js'])
.pipe(mocha({
compilers: {
js: babel
}
})
})
However if the unit test is broken I would like to exit the whole gulp workflow. How can I do this?
You could try to just kill the process and restarting it when you want? Prehaps i do not fully understand your question, if so please ellaborate.
In cmd where you run the gulpscript you can press CTRL + C, and than Y to affirm. This stops the current script.
Mocha runs the tests. Gulp simply groups files, folder locations and pipe invokes mocha with this grouped information. What you are asking for is for a mechanism for mocha to communicate back to gulp the test results instead of its stdout if I read your question correctly. gulp automatically exits when mocha exits but if it does not then either you have a watch task or there is a allback in your gulp file that has not been resolved or this issue - [https://github.com/sindresorhus/gulp-mocha/issues/1][1]
You can use
.on('error', process.exit.bind(process, 1))
to check if the process exits
Or, if it is a callback issue, resolve the call with a done()
gulp.task('taskname', function (done) {
gulp.src('test/testfile.js')
.pipe(gulpmocha(),setTimeout(function() {
done(null);
}, 5000))
.on('error', process.exit.bind(process, 1))
});
Related
One of my tests is failing intermittently when running the whole suite, but it doesn't fail when running it by itself.
I created a very basic repository with a vanilla application that reproduces the issue:
https://github.com/juanazam/ember-cli-test-issue.
Basically, I created a component with a text field and a button. The button is disabled while the text is empty.
The issues happens when two tests use the fillIn helper on the input.
Here is the testing code taken from the vanilla app:
test('test 1', function(assert) {
visit('/');
fillIn('input[type=text]', "Algo");
andThen(function() {
assert.equal(currentRouteName(), "index");
});
});
test('test 2', function(assert) {
visit('/');
andThen(function() {
assert.ok(find('input[type=submit]').is(':disabled'));
});
fillIn('input[type=text]', "Algo");
andThen(function() {
assert.ok(!find('input[type=submit]').is(':disabled'));
});
});
As you can see test 1 only fills the input but doesn't do anything with it. The second test tests if the button is disabled.
Test 2 fails intermittently when running the whole suite. If you run ember test -s it fails, if you reload the browser tab (re running the whole suite without restarting the server process) it passes. The same behavior happens with multiple runs (one run fails, the next succeeds).
I didn't create a twiddle reproduction case because the test runner doesn't behave the same way.
With you app:
ember test will always fail.
ember test --filter 'test 1' will always pass.
ember test --filter 'test 2' will always pass.
If you split your 2 test functions into different acceptance tests ember test will always pass.
Qunit in the browser tries to execute failing tests first. (I think to reduce the time until the most interesting failing tests are executed.). With ember -s your tests are always executed in order and tests are failing (I guess test 2 failes because test1 already filled your input and it is not initially disabled as expected).
When reloading qunit in browser after the first failed test, the failing test2 is executed first (and passes).
Also have a look on https://dockyard.com/blog/2014/04/17/ember-object-self-troll . There might be a problem within your component definition leading to the unexpectedly filled input in test2.
I'm trying to run some Jasmine tests in Karma but the tests are failing because it's saying that it ran 0 of 0 tests. Can someone tell me what I'm doing wrong?
The async request mock fires and hits the callback. Even when I go to the debugger, it says 2 tests completed in the debugger, but failing in the console. What gives?
describe('User Info Tests:', function () {
describe('Fetch User Info:', function () {
it("User Name should match", function(done) {
// mock async request
getUserProfile(1, 2, function (userProfile) {
var match = userProfile.displayName === 'Unit Test User';
expect(match).toBeTruthy();
done();
}, function (msg) {
done();
throw msg;
});
});
});
});
See the screenshot below of the debug console of the tests running. You will see the tests ran with a status of SUCCESS.
So the problem was I wasn't including the karam-requirejs plugin in the karam.conf.js file. Apparently it doesn't want you to include your own copy of require.js in the files collection. Once I added that plugin in, everything just worked.
frameworks: ['jasmine-jquery', 'jasmine', 'requirejs'],
plugins: [
'karma-phantomjs-launcher',
'karma-chrome-launcher',
'karma-jasmine-jquery',
'karma-jasmine',
'karma-requirejs'
],
Make sure the karma-requirejs plugin is actually installed through npm and in your package.json as well!
Is there any command for karma-jasmine unit-test to stop the test when it encounters the first test fail. For example, in python the command is like:
py.test -x # stop after first failure
py.test --maxfail=2 # stop after two failures
Currently I am using node_modules/karma/bin/karma start that run all the tests and stops only after everything is executed
This would require creating a custom reporter, or changing the reporter in the karma-jasmine adapter to stop on spec failure as such:
this.specDone = function (specResult)
{
var failure = specResult.failedExpectations.length;
if (failure)
{
suiteDone();
jasmineDone();
}
}
References
jasmine.io: custom_reporter.js
karma-jasmine source: adapter.js
Jasmine Issue #842: Async reporter hooks
Protractor Issue #1938: Find a good pattern for waiting for Jasmine Reporters
Alternatively you can just tell jasmine you want to run a specific Spec or Specs in a folder so you only are testing a subset of your tests and not running all in your suite.
I want to do a simple spec that involve DOM manipulate that phantomJS is working:
/*global describe it */
'use strict';
(function () {
describe('DOM Tests', function () {
var strong = 'Viva!';
var text = document.getElementById('test').innerHTML;
console.log(text);
it("is in DOM", function(){
expect(strong).equal('Viva!');
});
});
})();
But after running grunt test
there was no assertion
Running "clean:server" (clean) task
Cleaning ".tmp"...OK
Running "haml:app" (haml) task
Running "coffee:dist" (coffee) task
File .tmp/scripts/app.js created.
Running "coffee:test" (coffee) task
Running "compass:dist" (compass) task
directory .tmp/styles/
create .tmp/styles/main.css
Running "compass:server" (compass) task
unchanged app/styles/main.scss
Running "connect:test" (connect) task
Starting connect web server on localhost:9000.
Running "mocha:all" (mocha) task
Testing index.htmlOK
>> 0 assertions passed (0s)
Done, without errors.
Here is the code repository, in case you need to check my code
I'm trying to troubleshoot a unit test issue.
I used to have a mostly working Maven -> PhantomJS -> Qunit setup, but it was unpredictable so I took it apart to try to fix it.
I upgraded the software:
Qunit: 1.11.0
PhantomJS: 1.8
Phantom Qunit Runner Latest: https://github.com/jquery/qunit/tree/master/addons/phantomjs
I see the web GUI working. It runs and passes all 102 tests. The console prints this:
$ phantomjs --disk-cache=false runner.js http://localhost/ui/dcx/test.html
$ Took 16ms to run 0 tests. 0 passed, 0 failed.
If I comment out the exit command in the runner, it prints the console output for QUnit.done multiple times.
$ phantomjs --disk-cache=false runner.js http://localhost/ui/dcx/test.html
$ PhantomJS successfully loaded a page
$ QUnit.done callback fired
$ Took 15ms to run 0 tests. 0 passed, 0 failed.
$ QUnit.done callback fired
$ Took 1840ms to run 102 tests. 102 passed, 0 failed.
$ QUnit.done callback fired
$ Took 1841ms to run 102 tests. 102 passed, 0 failed.
$ QUnit.done callback fired
$ Took 1842ms to run 102 tests. 102 passed, 0 failed.
$ QUnit.done callback fired
$ Took 1848ms to run 102 tests. 102 passed, 0 failed.
$ ^C
$
Looks to me like the Qunit.done callback is getting executed too soon, then multiple times.
Anyone know why that callback fires?
My test inclusions and login delay might be relevant. I use AMD modules to define tests and curl.js to bring them in. Nothing happens until the security login does:
curl(['dolla'], function($){
$.ajax({
type: 'POST',
url: '/svc/j_spring_security_check',
data: {
j_username: '7',
j_password: '7'
},
success: function() {
loadTests()
}
});
})
var loadTests = function () {
curl([
// Unit tests
'dcx/dataControls/activity.test'
, 'dcx/dataControls/eventList.test'
, 'dcx/dataControls/mapViewer.view.test'
, 'dcx/pages/deviceDetails.view.test'
, 'dcx/pages/login.test'
, 'dcx/pages/nodeProfiles.test'
, 'dcx/pages/settings.view.test'
], function() {}, function(ex) { throw new Error(ex) })
})
EDIT:
I'm down to a root cause, I think.
If you include QUnit on a blank page, it calls QUnit.begin and QUnit.done right away.
I need to delay execution of Qunit until after the security login is successful and curl has brought in my unit tests. Is there a way to delay the start of QUnit, but still keep the Qunit object available? I can't use stop() because there are many async tests that will call start().
Found the answer. You can configure QUnit to not start, then start it manually when all your tests are loaded. This prevents the duplicate calling of Qunit.done which is the root cause of this issue.
http://forum.jquery.com/topic/are-qunit-and-requirejs-compatible#14737000001967123
This is one way to do it--modify the runner to not exit if there are not test results.
https://gist.github.com/SimpleAsCouldBe/5059623
This doesn't work though--Qunit.done fires whenever the test stack is cleared. In an asynchronously loaded environment like Curl/Require.js, this can happen any time.
This is if you don't want to use require. For example, in a browser context maybe.
Having spent ages to find a method to turn loading of scripts (and stylesheets) into Promises (see here), I then found big problems with QUnit test suites starting to run before all these had loaded. Typically a handful of tests, at the start, would complain than a certain variable or class was undefined, although later tests wouldn't have that difficulty.
You can stop automatic starting by going like this:
QUnit.config.autostart = false;
... seemingly just putting it in one of several files will suffice.
To start the QUnit tests, then, you have to go QUnit.start();. But, understandably perhaps, you can't execute this from inside any code which is being run by a QUnit test. This gets complicated. In the end I did this in my app-starting code:
await this.loadInjectedFile( GLOBAL_SCRIPT );
await this.loadInjectedFile( DBFORM_SCRIPT );
await this.loadInjectedFile( UDV_SCRIPT );
await this.loadInjectedFile( REACTIVITY_SCRIPT );
console.log( '... injected files loaded' );
// to allow QUnit to start testing
window.QUnitGreenLight = true;
... strictly speaking a naughty thing to do (allowing test-related code to sneak into your app code). A more compartmentalised approach could probably be found.
Then, inline in the HTML file from where you launch your testing:
<script>
const tryToStartTesting = function(){
setTimeout( function(){
if( window.QUnitGreenLight ){
QUnit.start();
}
else {
console.log( 'QUnit green light not yet given!' );
tryToStartTesting();
};
}, 10 );
};
tryToStartTesting();
</script>
... in practice it seems to take maybe a few hundredths of a second before the green light is given.
A bit scrappy, perhaps, but it seems to work.