Ember concurrency timeout hanging in qunit - ember.js

In Ember I have a component that starts a never-ending poll to keep some data up to date. Like so:
export default Component.extend({
pollTask: task(function * () {
while(true) {
yield timeout(this.get('pollRate'));
this.fetchSomeData();
}
}).on('init')
})
This causes a preexisting acceptance test to get stuck in this task and run forever, even though it should be run asynchronously. The test looks like this:
test('my test', async function(assert) {
mockFindRecord(make('fetched-model'));
await visit('/mypage'); // gets stuck here
await typeIn('input', 'abc123');
assert.ok(somethingAboutThePage);
});
I thought at first that I had mocked the request wrong and that the test was just timing out, but it was in fact correctly polling data. Removing this task makes the acceptance test finish normally.
Testing this manually seems to work fine, and nothing gets stuck. Why is this happening and what is the right way to address this?
Saw Unit testing ember-concurrency tasks and yields but it doesn't really help since it only deals with unit tests.

You're not doing anything wrong and this is a common gotcha with ember-concurrency. Ember-concurrency's timeout() function relies on Ember.run.later() to create the timeout and fortunately or unfortunately, Ember's test suite is aware of all timers created with Ember.run.later() and will wait for all timers to settle before letting the test continue. Since your code is using an infinite loop your timers will never settle so the test hangs. There's a nice little section about testing asynchronous code in the Ember guides here.
There's a section in the ember-concurrency docs about this exact problem here. I recommend you look through it to see their recommendations on how to tackle this although it seems as if there's no real elegant solution at the time.
The quickest and probably easiest way to get this to not hang would be to throw in a check to see if the app is being tested (gross, I know):
pollTask: task(function * () {
while(true) {
yield timeout(this.get('pollRate'));
this.fetchSomeData();
if (Ember.testing) return; // stop polling to prevent tests from hanging
}
}).on('init')
You can also try to throw in a call to Ember.run.cancelTimers() in your tests/helpers/start-app.js file (another suggestion in that section):
window._breakTimerLoopsId = Ember.run.later(() => {
Ember.run.cancelTimers();
}, 500);
But it doesn't seem to appear in the API docs so I personally wouldn't rely on it.

Related

Is it considered good practice to use expect.assertions in tests with Jest? [duplicate]

I've found a lot of this sort of thing when refactoring our Jest test suites:
it('calls the API and throws an error', async () => {
expect.assertions(2);
try {
await login('email', 'password');
} catch (error) {
expect(error.name).toEqual('Unauthorized');
expect(error.status).toEqual(401);
}
});
I believe the expect.assertions(2) line is redundant here, and can safely be removed, because we already await the async call to login().
Am I correct, or have I misunderstood how expect.assertions works?
expect.assertions is important when testing the error scenarios of asynchronous code, and is not redundant.
If you remove expect.assertions from your example you can't be confident that login did in fact throw the error.
it('calls the API and throws an error', async () => {
try {
await login('email', 'password');
} catch (error) {
expect(error.name).toEqual('Unauthorized');
expect(error.status).toEqual(401);
}
});
Let's say someone changes the behavior of login to throw an error based on some other logic, or someone has affected the mock for this test which no longer causes login to throw. The assertions in the catch block won't run but the test will still pass.
Using expect.assertions at the start of the test ensures that if the assertions inside the catch don't run, we get a failure.
This is from Jest documentation:
Expect.assertions(number) verifies that a certain number of assertions
are called during a test. This is often useful when testing
asynchronous code, in order to make sure that assertions in a callback
actually got called.
So to put in other words, expect.assertions makes sure that the n number of assertions are made by the end of the test.
It's good to use it especially when writing a new tests, so one can easily check that correct assertions are made during the test. Async tests often pass because the intended assertions were not made before the test-runner (Jest,Mocha etc.) thought the test was finished.
I think we are missing the obvious here.
expect.assertions(3) is simply saying ...
I expected 3 expect statements to be called before the test times out. e.g.
expect(actual1).toEqual(expected1);
expect(actual2).toEqual(expected2);
expect(actual3).toEqual(expected3);
This timing out business is the reason to use expect.assertions. It would be silly to use it in a purely synchronous test. At least one of the expect statements would be found in a subscribe block (or other async block) within the spec file.
To ensure that the assertions in the catch block of an async/await test are adequately tested, expect.assertions(n) must be declared as shown in your code snippet. Such declaration is unnecessary for async/await tests without the catch block.
It seems quite unintuitive but it is simply the way it is. Perhaps, for certain reasons well deep within the javascript runtime, the test environment can detect when an await'ed' promise successfully resolved but cannot detect same for await'ed' promises that failed to resolve. The creators of the test environment would likely know verbatim why such is the case.
I have to admit that apart from error testing, I find it challenging to see a real use for expect.assertions. The above snippet can be changed to the following with the same guarantee but I think it reads more naturally and doesn't require me to count how many time I call expect. This is especially error-prone if a test if complex:
it('calls the API and throws an error', async () => {
try {
await login('email', 'password');
fail('must throw')
} catch (error) {
expect(error.name).toEqual('Unauthorized');
expect(error.status).toEqual(401);
}
});

Ember 3.0 acceptance redirect test hangs forever

I have a simple acceptance test written in the modern RFC 268 format for Ember 3.0.
The test is for a page where, if the user is unauthenticated, the URL immediately redirects to /login.
...
module('Acceptance | index', function (hooks) {
setupApplicationTest(hooks);
test('visiting / logged out', async function(assert) {
assert.expect(1);
// Test hangs here forever.
await visit('/');
assert.equal(currentURL(), '/login');
});
});
This test worked great using the older format with moduleForAcceptance.
Unfortunately, this test hangs forever in Ember 3.0. Am I missing something here? Is there a better way to test a redirect?
There are no errors in the console, and the addition of some console.log statements show that the await is where the test hangs.
I found the reason why this was failing. I have an Ember mixin that I use to enhance all of my routes. The mixin checks for whether or not a user is authenticated, and redirects to /login as needed.
export default Mixin.create({
session: service(),
beforeModel() {
this._super(...arguments);
return new EmberPromise((resolve) => {
if (authenticated) {
resolve();
return;
}
this.transitionTo('login');
});
}
});
You'll notice that I am not resolving if authenticated is falsey. That worked fine with my app and the test syntax in 2.18.
The docs say the following regarding that hook I am overriding in my mixin.
returns Any | Promise
if the value returned from this hook is a promise, the transition will pause until the transition resolves. Otherwise, non-promise return values are not utilized in any way.
To me, the bit about "non-promise return values" implies that I should be able to do what I'm doing. Especially considering this worked in 2.18, but I wonder if this was one of those "wow, how did that ever work in the first place" scenarios. Clearly this syntax isn't working in 3.0. since the transition pauses forever when testing.
The answer for me was to ensure I always resolve/reject something. In this case, I had to add an explicit reject() so that the promise chain doesn't hang.
export default Mixin.create({
session: service(),
beforeModel() {
this._super(...arguments);
return new EmberPromise((resolve) => {
if (authenticated) {
resolve();
return;
}
this.transitionTo('login');
reject();
});
}
});
My test was fine. It was the mixin that needed updating in order to work properly with Ember 3.0 and the latest testing syntax.
The issue was not what your beforeModel hook resolves if user is authenticated but that your Promise does not resolve at all. You don't have to return a Promise in beforeModel hook but if you return one it will block the transition until the Promise is resolved. Since it's not clear how ember should react if another transition is called while current transition is blocked (not resolved/rejected promise), resolving or rejecting is correct behavior. Please have in mind that in a Promise return does not have any other meaning than ending your execution. It does not resolve or reject your Promise.
Another reason visit() could hang is that it waits for timers like Ember.run.later() to resolve, causing a non-obvious block somewhere in the application.
AlphaGit on github summarized the issue with an example, saying:
Most of the actions that Ember.testing executes (like visit) will
append to a promise that gets executed action after action in the
right order. In order to pass to the next action, Ember.testing makes
sure that there is nothing pending, so that the step can be considered
complete and move forward.
Along with the things that are tested for, pending AJAX requests are
verified, and also scheduled timers. These timers may arise from, you
guessed it, Ember.run.later calls. If for any reason you would have in
your code periodic Ember.run.later methods (so that one is always
waiting to be excuted), it's likely that you'll face this issue.
I've faced it myself in a similar scenario: My server returns a OAuth
access token with 100 hours until expired, so ember-simpleAuth
registers a call close to the expiration time with Ember.run.later to
refresh the token. This will, however, prevent the test from moving
along. My specific situations has been fixed in further versions but
any similar behavior will reproduce the issue (which is likely a
conclusion of the current design of Ember.testing).
Here are a couple other examples of users running into similar issues:
https://stackoverflow.com/a/58526993/3257984
https://stackoverflow.com/a/27887807/3257984

Ember/emberfire run loop acceptance test

So my acceptance test keeps on destroying itself before my promise finishes. I'm aware that I need to wrap my promise in Ember run loop but I just can't get it to work. Here's how my component looks:
export default Ember.Component.extend({
store: Ember.inject.service(),
didReceiveAttrs() {
this.handleSearchQueryChange();
},
/**
* Search based on the searchQuery
*/
handleSearchQueryChange() {
this.get('store').query('animals', {
orderBy: 'name',
startAt: this.attrs.searchQuery
}).then(searchResults => {
this.set('searchResults', searchResults);
});
}
});
I've already tried wrapping this.handleSearchQueryChange(), this.get('store').query... and this.set('searchResults', searchResults) in a run loop but still, the acceptance test just doesn't wait for store.query to finish.
One thing to note that this store query performs a request on a live Firebase back-end.
I'm currently using Pretender to mock the data and solve this issue. But I'd like to solve it through Ember.run as well. Anyone care to provide a solution?
It sounds like your problem may have the same cause as the errors I've been seeing
tl;dr
To work around this issue, I've been using a custom test waiter. You can install it with ember install ember-cli-test-model-waiter (for Ember v2.0+) and it should just make your test work without any further setup (if not, please file a bug).
Longer answer:
The root cause of this problem is that the ember testing system doesn't know how to handle Firebase's asynchronicity. With most adapters, this isn't a problem, because the testing system instruments AJAX calls and ensures they have completed before proceeding, but this doesn't work with Firebase's websockets communication.
The custom test waiter I mentioned above works by waiting for all models to resolve before proceeding with the test, and so should work with any non-AJAX adapter.

One passing Ember integration test breaks the next one: bad teardown block?

I have these two Ember integration tests, A and B. (I have a lot more, but in debugging this I have literally removed every other test in order to isolate the problem. There are 9 tests in the same file as A and I commented the other 8.) If A runs before B, B will fail. If B runs by itself, or before A, it will pass.
From this description it seems pretty clear that A is doing something to the test environment that screws up B. After liberally salting the tests and the production code involved with log messages, however, I'm no closer to figuring out what's up, and I'm hoping someone else can spot if there's an obvious issue.
Right now I'm looking closely at the afterEach blocks in both tests. Here's an outline of what the beforeEach and afterEach blocks look like for test A:
beforeEach: function() {
server = new Pretender(function() {
// Pretender setup removed for brevity
});
App = startApp();
},
afterEach: function() {
server.shutdown();
Ember.run(App, App.destroy);
}
That afterEach is pretty much the stock ember-cli code, but it baffles me a bit. The documentation on Ember.run() suggests it should get a function as an argument, but we're not giving it one here, so I'm not sure how that works. And, should the Pretender shutdown() call be inside the Ember.run (or in its own Ember.run)?
The versions, for the record: ember-cli 0.2.0, Ember 1.10.1.
ETA: The issue goes away when I update to ember-cli 0.2.3 and Ember 1.11.3. Now if only I could figure out the other failing tests we have with that update...
Your setup and teardown looks fine. They are commonly used and are properly defined.
However, there is (still) open issue on ember-qunit about not tearing down the app properly - take a look here to see the progress.
As you said, it does not happen in Ember 1.13.

Errors from debounced function calls in testing ember application

I'm testing an relatively large Ember application (http://wheelmap.org/map) with QUnit and having problems with debounced calls e.g. changing the url to have a permalink of a map view inside the app or doing a manual AJAX request while testing.
I followed the documentation at http://emberjs.com/guides/testing/integration/
Now when I reset the application state by calling App.reset() in the module setup it resets all bindings, etc. to variables and dependant controllers.
module('Map', {
setup: function() {
App.reset();
}
});
This seems to be good to have a clean working environment, but leads to errors where variables are accessiable by Ember.set and Ember.get e.g. this.get('controllers.toolbar'):
Cannot call method 'set' of null
So the first test allways runs great, but further tests break because of debounced function calls from the first test. So what I think I have to do is stop this debounced calls somehow.
Other options would be checking if all needed variables are set in this function calls. But this seems to be cumbersome when adding conditions only for testing.
What do you think?
Thank you in advance!
I found the answer by searching through the RunLoop source files:
Ember.run.cancelTimers()
It's not part of the documentation. Maybe a problem of poor documentation or not beeing part of the public API.
Now I just call it in the module test teardown function:
module('Map', {
setup: function() {
// ...
},
teardown: function() {
Ember.run.cancelTimers()
}
});
We ran into a similar problem and decided to disable debounce during testing.
You can check if in testing mode using if(Ember.testing){...}.