I have been trying to configure offline unit tests for polymer web components that use the latest release of Firebase distributed database. Some of my tests are passing, but others—that look nigh identical to passing ones—are not running properly.
I have set up a project on github that demonstrates my configuration, and I'll provide some more commentary below.
Sample:
https://github.com/doctor-g/wct-firebase-demo
In that project, there are two suites of tests that work fine. The simplest is offline-test, which doesn't use web components at all. It simply shows that it's possible to use the firebase database's offline mode to run some unit tests. The heart of this trick is the in the suiteSetup method shown below—a trick I picked up from nfarina's work on firebase-server.
suiteSetup(function() {
app = firebase.initializeApp({
apiKey: 'fake',
authDomain: 'fake',
databaseURL: 'https://fakeserver.firebaseio.com',
storageBucket: 'fake'
});
db = app.database();
db.goOffline();
});
All the tests in offline-test pass.
The next suite is wct-firebase-demo-app_test.html, which test the eponymous web component. This suite contains a series of unit tests that are set up like offline-test and that pass. Following the idea of dependency injection, the wct-firebase-demo-app component has a database attribute into which is passed the firebase database reference, and this is used to make all the firebase calls. Here's an example from the suite:
test('offline set string from web component attribute', function(done) {
element.database = db;
element.database.ref('foo').set('bar');
element.database.ref('foo').once('value', function(snapshot) {
assert.equal(snapshot.val(), 'bar');
done();
});
});
I have some very simple methods in the component as well, in my attempt to triangulate toward the broken pieces I'll talk about in a moment. Suffice it to say that this test passes:
test('offline push string from web component function', function(done) {
element.database = db;
let resultRef = element.pushIt('foo', 'bar');
element.database.ref('foo').once('value', function(snapshot) {
assert.equal(snapshot.val()[resultRef.key], 'bar');
done();
});
});
and is backed by this implementation in wct-firebase-demo-app:
pushIt: function(at, value) {
return this.database.ref(at).push(value);
},
Once again, these all pass. Now we get to the real quandary. There's a suite of tests for another element, x-element, which has a method pushData:
pushData: function(at, data) {
this.database.ref(at).push(data);
}
The test for this method is the only test in its suite:
test('pushData has an effect', function(done) {
element.database = db;
element.pushData('foo', 'xyz');
db.ref('foo').once('value', function(snapshot) {
expect(snapshot.val()).not.to.be.empty;
done();
});
});
This test does not pass. While this test is running, the console comes up with an error message:
Your API key is invalid, please check you have copied it correctly.
By setting some breakpoints and walking through the execution, it seems to me that this error comes up after the call to once but before the callback is triggered. Note, again, this doesn't happen with the same test structure described above that's in wct-firebase-demo-app.
That's where I'm stuck. Why do offline-test and wct-firebase-demo-app_test suites work fine, but I get this API key error in x-element_test? The only other clue I have is that if I copy in a valid API key into my initializeApp configuration, then I get a test timeout instead.
UPDATE:
Here is a (patched-together) image of my console log when running the tests.:
To illustrate the issue brought up by tony19 below, here's the console log with just pushData has an effect in x-element_test commented out:
The offline-test results are apparently false positives. If you check the Chrome console, offline-test actually throws the same error:
The error doesn't affect the test results most likely because the API key validation occurs asynchronously after the test has already completed. If you could somehow hook into that validation, you'd be able to to catch the error in your tests.
Commenting out all tests except for offline firebase is ok shows the error still occurring, which points to suiteSetup(). Narrowing the problem down further by commenting 2 of the 3 function calls in the setup, we'll see the error is caused by the call to firebase.initializeApp() (and not necessarily related to once() as you had suspected).
One workaround to consider is wrapping the Firebase library in a class/interface, and mocking that for unit tests.
Related
Has someone worked on unit testing ag-grid components in Angular 2?
For me, this.gridOptions.api remains undefined when the test cases run.
Sorry to be a little late to the party but I was looking for an answer to this just a couple of days ago so wanted to leave an answer for anyone else that ends up here. As mentioned by Minh above, the modern equivalent of $digest does need to be run in order for ag-grid api's to be available.
This is because after the onGridReady() has run you have access to the api's via the parameter, looking like so. This is run automatically when a component with a grid is initialising. Providing it is defined in the grid (gridReady)="onGridReady($event)"
public onGridReady(params)
{
this.gridOptions = params;
}
This now means you could access this.gridOptions.api and it would be defined, you need to re-create this in your test by running detectChanges(). Here is how I got it working for my project.
fixture = TestBed.createComponent(TestComponent);
component = fixture.componentInstance;
fixture.detectChanges(); // This will ensure the onGridReady(); is called
This should inturn result in .api being defined when running tests. This was Angular 6.
Occasionally the test may have to perform an await or a tick:
it('should test the grid', fakeAsync( async () => {
// also try tick(ms) if a lot of data is being tested
// try to keep test rows as short as possible
// this line seems essential at times for onGridReady to be processed.
await fixture.whenStable();
// perform your expects...after the await
}));
If you are using ag-grid enterprise make sure to include in your test file import 'ag-grid-enterprise'; otherwise you will see console errors and gridReady will never be called:
Row Model "Server Side" not found. Please ensure the ag-Grid Enterprise Module #ag-grid-enterprise/server-side-row-model is registered.';
It remains undefined because the event onGridReady is not invoked yet. Im not sure about Angular 2 because im using angularjs and have to do $digest in order to invoke onGridReady.
Prior to firebase 3 update our acceptance test have been running without any issues. We use the following in our beforeTest and afterTest
moduleForAcceptance('Acceptance | Dashboard | Items | Library | New', {
beforeEach() {
stubFirebase();
var ref = createOfflineRef(basicDataRef, 'https://MY-APP.firebaseio.com');
replaceAppRef(this.application, ref);
stubValidSession(this.application, {uid: 'xxxx'});
},
afterEach() {
unstubFirebase();
}
});
basicDataRef is a fixture for the test. The above code allows my to mock session following the test-helper in torii library to allow my application to correctly obtain the data needed as my firebase hieararchy is as follows:
/
+--uid
+--profile
+--otherdata
I am not testing for permission rules, just interaction to save/edit data in the application, and this has worked OK prior to firebase 3 migration. After version 3 all my test returns the following:
actual: >
false
expected: >
true
stack: >
at http://localhost:7357/assets/test-support.js:4130:12
at exports.default._emberTestingAdaptersAdapter.default.extend.exception (http://localhost:7357/assets/vendor.js:49473:7)
at onerrorDefault (http://localhost:7357/assets/vendor.js:41461:24)
at Object.exports.default.trigger (http://localhost:7357/assets/vendor.js:62212:11)
at http://localhost:7357/assets/vendor.js:63463:40
at Queue.invoke (http://localhost:7357/assets/vendor.js:10415:16)
message: >
Error: permission_denied at /xxxx/profile: Client doesn't have permission to access the desired data.
I always thought the createOfflineRef in emberfire allows us to bypass rules checking. the fact that it keeps returning permission_denied is quite perplexing. Maybe i need to re-engineer the test? Or I approach this wrongly all this time? Any input is greatly appreciated
Got to the bottom of this, and I guess I'll answer my own questions in case somebody else experience the same issue as I have.
the new firebase InitializeApp method has an additional optional parameter called name. By default, Emberfire service sets this name to be:
export const DEFAULT_NAME = '[EmberFire default app]';
However the Emberfire test helper to create firebase offline ref stubs the firebase instance with a different instance name to be:
export const DEFAULT_NAME = '[EmberFire offline test app]';
This cause my test to fail with permission denied, as the acceptance test is attempting to connect to the '[EmberFire default app]' and the stubbed offline reference is called something else.
Creating my own create-offline-ref helper substituting the DEFAULT_NAME to '[EmberFire default app]' solves the problem. I'm not sure as to what is the best practice for acceptance test as the change seems deliberate on emberfire.
I've set up an error substate route/controller/template according to http://guides.emberjs.com/v2.2.0/routing/loading-and-error-substates/. Manually browsing my app, I can trigger error conditions and get directed to the substate. Confirmed with Ember Inspector.
I'd like to automatically test the substate. However, Ember CLI's test runner fails any test when a route's model hook rejects. In other words, the test fails before I can navigate to the error substate.
How can I automatically test my error substate?
Ember: 2.2.0
Ember CLI: 1.13.13
Unfortunately it doesn't seem to be easy to do this in a clean manner.
In its internal tests, Ember uses bootApplication to the route which errors (see github) and is able to directly catch the error. Unfortunately if you try and do any form of try/catch or then/catch around a call to visit in your tests you will find it fails.
When you visit a link which results in an error substate from your acceptance test then Ember's defaultActionHandlers.error gets fired. By design it is not meant to be overridable. It calls logError which calls Ember.default.Logger.error.
So to test this substate we need to temporarily overwrite that method. We can also peek inside the ember container to access the currentRouteName like so (using sinon for the spying):
test('when there is an API error an error message is shown', function(assert) {
const emberLoggerError = Ember.Logger.error;
Ember.Logger.error = sinon.spy();
visit('/users/');
andThen(() => {
// This could be nicer and less private with `getOwner`
let { currentRouteName } = this.application.__container__.lookup('router:main');
assert.equal(currentRouteName, 'users.index_error', 'The current route name is correct');
assert.equal(Ember.Logger.error.callCount, 1, 'The error logger was called');
// Restore the Ember.Logger
Ember.Logger.error = emberLoggerError;
});
});
Things can get even more complicated though. If your visit happens inside a Promise (it did in our case because we were using ember-page-object for our tests) then you have more to deal with...
In a separate loop onerrorDefault of RSVP is triggered which calls Test.adapter.exception AND Ember.default.Logger.error (again!) - passing the stack. So in this case you need to stub and spy on Test.adapter.exception and expect Ember.default.Logger.error to have been called twice!
I'm testing an relatively large Ember application (http://wheelmap.org/map) with QUnit and having problems with debounced calls e.g. changing the url to have a permalink of a map view inside the app or doing a manual AJAX request while testing.
I followed the documentation at http://emberjs.com/guides/testing/integration/
Now when I reset the application state by calling App.reset() in the module setup it resets all bindings, etc. to variables and dependant controllers.
module('Map', {
setup: function() {
App.reset();
}
});
This seems to be good to have a clean working environment, but leads to errors where variables are accessiable by Ember.set and Ember.get e.g. this.get('controllers.toolbar'):
Cannot call method 'set' of null
So the first test allways runs great, but further tests break because of debounced function calls from the first test. So what I think I have to do is stop this debounced calls somehow.
Other options would be checking if all needed variables are set in this function calls. But this seems to be cumbersome when adding conditions only for testing.
What do you think?
Thank you in advance!
I found the answer by searching through the RunLoop source files:
Ember.run.cancelTimers()
It's not part of the documentation. Maybe a problem of poor documentation or not beeing part of the public API.
Now I just call it in the module test teardown function:
module('Map', {
setup: function() {
// ...
},
teardown: function() {
Ember.run.cancelTimers()
}
});
We ran into a similar problem and decided to disable debounce during testing.
You can check if in testing mode using if(Ember.testing){...}.
I have a small problem, I have created some Selenium tests. The problem is I can't order the testcases I have created. I know unit testing should not be ordered but this is what I need in my situation. I have to follow these steps: login first, create a new customer, change some details about the customer and finally log out.
Since there is no option to order unit tests in NUnit I can't execute this.
I already tried another option, to create a unittest project in Visual Studio, because Visual Studio 2012 has the ability to create a ordered unit test. But this is not working because I can't run a unit test while I am running my ASP.NET project. Another solution file is also not a good option because I want to verify my data after it has been submitted by a Selenium test.
Does someone of you have another solution to solve my problem?
If you want to test all of those steps in a specific order (and by the sounds of it, as a single session) then really it's more like an acceptance test you are talking about; and in that case it's not a sin to write more complex test methods and Assert your conditions after each step.
If you want to test each step in true isolation (a pure unit test) then each unit test must be capable of running by itself without any reference to any other tests; but when you're testing the actual site UI itself this isn't really an option for you.
Of course if you really you want to have every single test somehow setup every single dependency without reference to any other actions (e.g in the last test you would need to fake the login token, your data layer will have to pretend that you added a new customer, etc. A lot of work for dubious benefit...)
I say this based on the assumption that you already have unit tests written for the server-side controllers, layers, models, etc, that you run without any reference to the actual site running in a browser and are therefore confident that the various back-end part of your site do what they are supposed to do
In your case I'd recommend more of a hybrid integration/acceptance test
void Login(IWebDriver driver)
{
//use driver to open browser, navigate to login page, type user/password into box and press enter
}
void CreateNewCustomer(IWebDriver driver)
{
Login(driver);
//and then use driver to click "Create Customer" link, etc, etc
}
void EditNewlyCreatedCustomer(IWebDriver driver)
{
Login(driver);
CreateNewCustomer(driver);
//do your selenium stuff..
}
and then your test methods:
[Test]
void Login_DoesWhatIExpect()
{
var driver = new InternetExplorerDriver("your Login URL here");
Login(driver);
Assert(Something);
}
[Test]
void CreateNewCustomer_WorksProperly()
{
var driver = new InternetExplorerDriver("your Login URL here");
CreateNewCustomer(driver);
Assert(Something);
}
[Test]
void EditNewlyCreatedCustomer_DoesntExplodeTheServer()
{
var driver = new InternetExplorerDriver("your Login URL here");
EditNewlyCreatedCustomer(driver);
Assert(Something);
}
In this way the order of the specific tests do not matter; certainly if the Login test fails then the CreateNewCustomer and EditNewlyCreatedCustomer tests will also fail but that's actually irrelevant in this case as you are testing an entire "thread" of operation