Dynamically loaded test scripts disrupts test execution order - unit-testing

I'm running a set of QUnit tests that use module level setup and teardown methods. I've noticed that using start() and stop() inside my tests appears to disrupt when these get called, which causes problems as certain items made available in my setup are not available to some tests that run.
Edit: I've noticed that this happens exclusively when I load my test scripts programmatically (I'm using a script loader: LABjs). I have modified the subject and content of this question accordingly. I am loading tests like this:
$LAB.script('/static/tests.js')
Still not sure why this happens.
Here's a sample of my test module:
module('Class Foo', {
setup: function() {
console.log('setup called');
},
teardown: function() {
console.log('teardown called');
}
});
test('Test1', function() {
stop();
console.log('test1');
ok(true);
start();
});
test('Test2', function() {
stop();
console.log('test2');
ok(true);
start();
});
test('Test3', function() {
stop();
console.log('test3');
ok(true);
start();
});
This yields the console output (note that setup is called twice, then not again):
setup called
test1
teardown called
(2)setup called
test3
teardown called
test2
teardown called
Remove the start/stop, or modifying my test files to not be loaded programatically (i.e.: using traditional tags):
test('Test3', function() {
console.log('test3');
ok(true);
});
Yields a more expected order of execution:
setup called
test1
teardown called
setup called
test2
teardown called
setup called
test3
teardown called
Am I misunderstanding something about how this should be functioning?

It appears that QUnit likes your test scripts to be loaded when it kicks itself off. This action is configurable, so I found the following setup worked to defer QUnit starting until all test scripts were available:
QUnit.config.autostart = false;
$LAB.script('/static/tests.js').wait(function() {
QUnit.start();
});
I'm still not sure why this happens so would be interested to see any answers in that regard (or I'll update this when I figure it out!) but this solution gets me by for now.

Related

Ember Mirage: route answered only if debugger is ran

I'm facing a weird issue with Ember mirage. I try to use it inside an integration test.
The code looks like this:
moduleForComponent('editors/steps/call-handler', 'Integration | Component | editors/steps/call handler', {
integration: true,
beforeEach: function () {
startApp();
this.server = startMirage();
...
},
afterEach: function () {
this.server.shutdown();
}
});
test('...', function (assert) {
...
this.server.post('/projects/12/actionwords/1/switch/', () => {
assert.ok(true, "switch route is called");
return {
... some JSON data
}
});
this.render(hbs`{{editors/steps/call-handler
...
}}`);
... and then some manipulation of the component
});
Now the test fails when I run it this way. The weird part is, if I place a debugger just before rendering the component, the test will pass.
I guess there are some timeout issues sooner or later that the debugger hides. Usually, I would only render the component in a callback to ensure the route has been correctly setup but this.server.post returns nothing not has any callbacks (except if I missed something in the doc).
I also tried setting the timing option (just in case) but it doesn't change anything (as expected).
I also tried to add an homemade sleep function to wait one second without the debugger and then the tests work fine. Of course if I can have a clean way to get them working, I'd prefer ;)
Did anyone face the same kind of issues and found a way to solve it ?
Best regards,
Vincent

Delay tests till Ember.run.later is done

I am trying to get some tests to pass for an ember addon. It was working fine until yesterday I added some code that runs later in the run loops using Em.run.next.
Here is what Im doing in my test.
visit('/').then(function() {
find('bm-select').click();
andThen(function() {
equal(1,1, 'yay');
});
});
The problem is when click is triggered, the later function is executed after andThen. By that time all my tests are done and it throws error. I am under the impression andThen should wait for all async stuff to finish.
This is what what my code looks like when click is triggered(focusOut event is triggered on click)
lostFocus: function() {
if(this.get('isOpen')) {
Em.run.later(this, function() {
var focussedElement = document.activeElement;
var isFocussedOut =
this.$().has(focussedElement).length === 0 && !this.$().is(focussedElement);
if(isFocussedOut) {
this.closeOptions({focus:false});
}
}, 0);
}
}.on('focusOut'),
You can see that it gives an error Uncaught TypeError: Cannot read property 'has' of undefined. This is from the focusOut method. By the time the function executes the components _state is 'destroying' and this.$() returns undefined.
I tried the wait helper and still I am not able to get the tests to work. How is this normally done.
I have extracted the tests to run in a bin. Here is the link to it.
After further debugging, the problem is one of tags 'bm-select' has it focusOut event triggered in the teardown method of testing. So by the time the run loop code executes the component is not inDOM.
I just added a hidden input field in the test app. Once I've run all the tests, I set focus to the hidden input field and use the wait test helper. Now all the run loop code is done by the time the tear down method is executed.

Uncaught Error: Assertion Failed: calling set on destroyed object

working in ember-cli testing. After all tests passed it returns extra two test with errors.
Uncaught Error: Assertion Failed: calling set on destroyed object
Source : '../dist/assets/vendor.js:13269'
this is one unit test configuration
import Ember from "ember";
import { test,moduleFor } from 'ember-qunit';
import startApp from '../helpers/start-app';
var App;
module('An Integration test',{
setup:function(){
App=startApp();
},
teardown: function() {
Ember.run(App, 'destroy');
}
});
This is either because in the result of a promise or any other deferred code you do not check the destroy status of an object, or because you didn't teardown something that has been setup and interact with DOM events or anything external to the core of Ember.
I used to have this especially on some jQuery plugins which I mapped to Ember, and during the tests the plugins were destroying too slowly and I was then either not using a run loop, or not checking the destroyed status of the Ember object I was manipulating.
You can do so with:
if ( !(obj.get('isDestroyed') || obj.get('isDestroying')) ) {
// do your destroying code setting stuff
}
Also think about destroying any jQuery plugins that might have been initialised in the code of your views (anything setup in didInsertElement should be teardown in willDestroyElement for example).
Ok i struggled with similar thing. So basically when you have "this.set()" inside a promise, it might happen that the promise takes too long to resolve, and the user already clicked away from that page, in this case you are trying to set something, that is already destroyed. I found the simplest solution to be just a simple check in the beginning of the promise.
if (this.isDestroyed) {
return;
}
this.set('...');
...
Edit: alternatively you can use Ember.trySet.
The issue is related to a promise not completely resolving and another test getting run immediately after.
You should give Ember Concurrency a try.
import { task, timeout } from 'ember-concurrency';
myFunction: task(function * () {
// do somethinng
yield timeout(1000); // wait for x milliseconds
// do something else
}).drop(),
I had a similar issue in an integration test. To resolve, in the integration test, I waited before performing the next action.
import wait from 'ember-test-helpers/wait';
wait().then(() => {
// perform action (which previously used to cause an exception)
});

Errors from debounced function calls in testing ember application

I'm testing an relatively large Ember application (http://wheelmap.org/map) with QUnit and having problems with debounced calls e.g. changing the url to have a permalink of a map view inside the app or doing a manual AJAX request while testing.
I followed the documentation at http://emberjs.com/guides/testing/integration/
Now when I reset the application state by calling App.reset() in the module setup it resets all bindings, etc. to variables and dependant controllers.
module('Map', {
setup: function() {
App.reset();
}
});
This seems to be good to have a clean working environment, but leads to errors where variables are accessiable by Ember.set and Ember.get e.g. this.get('controllers.toolbar'):
Cannot call method 'set' of null
So the first test allways runs great, but further tests break because of debounced function calls from the first test. So what I think I have to do is stop this debounced calls somehow.
Other options would be checking if all needed variables are set in this function calls. But this seems to be cumbersome when adding conditions only for testing.
What do you think?
Thank you in advance!
I found the answer by searching through the RunLoop source files:
Ember.run.cancelTimers()
It's not part of the documentation. Maybe a problem of poor documentation or not beeing part of the public API.
Now I just call it in the module test teardown function:
module('Map', {
setup: function() {
// ...
},
teardown: function() {
Ember.run.cancelTimers()
}
});
We ran into a similar problem and decided to disable debounce during testing.
You can check if in testing mode using if(Ember.testing){...}.

Handling several event listeners

Update: here's a fiddle of my problem. The tests pass once, and fail the next time:
http://jsfiddle.net/samselikoff/hhk6u/4/
The problem is departments has events.on("userSet:company"), so both variables respond to the event.
This is a general question about unit testing. In my app, a certain event is fired, and several other pieces of my app listen for this event. I'd like to unit test each piece separately, since they are performing different functions; but to do this, I have to fire off the event in each test.
This causes problems, since the first test must fire off the event, triggering the listeners in the other tests. How can I keep my tests atomic while still testing multiple event listeners?
(I am using QUnit, but I think this is a more general unit-testing question).
Answer:
Jeferson is correct. One easy way to solve this, is to use events.once instead of events.on. This way you clean up your events from each test.
All your calls to async methods should be tested using "asyncTest" methods and making sure you wrap your calls in other functions that calls QUnit.start() when the assertions data are ready to be collected and analyzed.
I updated your JSFiddle with working code: http://jsfiddle.net/hhk6u/8/
The new code is:
QUnit.config.autostart = false;
QUnit.config.testTimeOut = 1000;
asyncTest('Some test that needs companies.', function() {
function getCompanies() {
var companies = new Companies();
ok(1);
start();
}
setTimeout(getCompanies, 500);
});
asyncTest('Some other async test that triggers a listener in companies.', function() {
var companies = new Companies();
events.trigger("userSet:company", { name: "Acme", id: 1 });
stop();
events.on('fetched:departments', function(response) {
console.log(response);
deepEqual(response, [1, 2, 3]);
start();
});
});
See my answer in this other question for more details:
Test fails then succeeds
Hope this helps you!