beforeEach fails in karma + Mocha with angular 4 upgrade - unit-testing

Question

Context
I have confirmed "green" builds on the master branch which is using angular-cli 1.0.0 and the old angular2 dependencies. My Objective is to upgrade from angular2 to angular4.
Post upgrade issue
The application works
The unit tests fail in karma using ng test and npm run test which are equivalent
Working configuration
specifically the package.json is as follows.
...
"dependencies": {
"#angular/common": "^2.4.0",
"#angular/compiler": "^2.4.0",
"#angular/core": "^2.4.0",
"#angular/forms": "^2.4.0",
"#angular/http": "^2.4.0",
"#angular/platform-browser": "^2.4.0",
"#angular/platform-browser-dynamic": "^2.4.0",
"#angular/router": "^3.4.0",
...
},
"devDependencies": {
"#angular/cli": "1.0.0",
"#angular/compiler-cli": "^2.4.0",
...
}
With the following test result
Chromium 53.0.2785 (Ubuntu 0.0.0): Executed 82 of 83 (skipped 1) SUCCESS (5.674 secs / 0.061 secs)
Angular 4 upgrade fails the tests
When I try to upgrade the angular4 dependencies with a very specific diff the application runs successfully, however I get failed/repeated tests for the upgrade like so Chromium 53.0.2785 (Ubuntu 0.0.0): Executed 127 of 83 (32 FAILED) (skipped 2) (7.521 secs / 0.101 secs)
What I find fascinating about the result is that there are 44 tests run in addition to the number of tests I actually have. It also appears that the only tests that fail are the duplicate tests that are running and they fail directly after they succeed. On investigation is seems that these tests would fail if their before each were not run
Chrome 56.0.2924 (Linux 0.0.0): Executed 67 of 83 (13 FAILED) (0 secs / 0.11 secs)
✔ will raise a houseCreated event
Chrome 56.0.2924 (Linux 0.0.0) HouseListComponent when valid input is provided (while in create mode) will raise a houseCreated event FAILED
AssertionError: expected { Object (country, state, ...) } to equal { Object (country, state, ...) }
at Context.<anonymous> (webpack:///src/app/housing/components/edit/house-edit.component.spec.ts:69:40 <- src/test.ts:26683:54)
at ZoneDelegate.invoke (webpack:///~/zone.js/dist/zone.js:365:0 <- src/polyfills.ts:1520:26)
at ProxyZoneSpec.Array.concat.ProxyZoneSpec.onInvoke (webpack:///~/zone.js/dist/proxy.js:79:0 <- src/test.ts:88673:39)
at ZoneDelegate.invoke (webpack:///~/zone.js/dist/zone.js:364:0 <- src/polyfills.ts:1519:32)
at Zone.run (webpack:///~/zone.js/dist/zone.js:125:0 <- src/polyfills.ts:1280:43)
at Context.<anonymous> (webpack:///~/zone.js/dist/mocha-patch.js:88:0 <- src/test.ts:88521:33)
at webpack:///~/zone.js/dist/mocha-patch.js:140:0 <- src/test.ts:88573:33
at ZoneDelegate.invokeTask (webpack:///~/zone.js/dist/zone.js:398:0 <- src/polyfills.ts:1553:31)
at Zone.runTask (webpack:///~/zone.js/dist/zone.js:165:0 <- src/polyfills.ts:1320:47)
at drainMicroTaskQueue (webpack:///~/zone.js/dist/zone.js:593:0 <- src/polyfills.ts:1748:35)
at ZoneTask.invoke (webpack:///~/zone.js/dist/zone.js:464:0 <- src/polyfills.ts:1619:25)
at timer (webpack:///~/zone.js/dist/zone.js:1540:0 <- src/polyfills.ts:2695:29)
Chrome 56.0.2924 (Linux 0.0.0): Executed 68 of 83 (14 FAILED) (0 secs / 0.111 secs)
✖ will raise a houseCreated event
broken configuration
...
"dependencies": {
"#angular/common": "^4.0.1",
"#angular/compiler": "^4.0.1",
"#angular/core": "^4.0.1",
"#angular/forms": "^4.0.1",
"#angular/http": "^4.0.1",
"#angular/platform-browser": "^4.0.1",
"#angular/platform-browser-dynamic": "^4.0.1",
"#angular/router": "^4.0.1",
...
},
"devDependencies": {
"#angular/cli": "1.0.0",
"#angular/compiler-cli": "^4.0.1",
...
}
Same tests on a different runner
In attempt to identify if the issue was the tests or the runner, I ran the exact same tests with the wallabyjs runner and I get No failing tests, 82 passing which is perfectly inline with master using the upgraded angular4 dependencies. This would seem to indicate that the issues is specific to karma and not the tests.
resources
Working master
Broken branch
Errors
There are a number of test errors, however they all seem to be related to beforeEach. The test module should not already be instantiated which seems to be a weird side effect from the repeating tests.
Chrome 56.0.2924 (Linux 0.0.0) car reducer "before each" hook for "will return array removes existing values when state is not empty" FAILED
Error: Cannot configure the test module when the test module has already been instantiated. Make sure you are not using `inject` before `TestBed.configureTestingModule`.
at TestBed.Array.concat.TestBed._assertNotInstantiated (webpack:///~/#angular/core/#angular/core/testing.es5.js:749:0 <- src/test.ts:25844:19) [ProxyZone]
at TestBed.Array.concat.TestBed.configureTestingModule (webpack:///~/#angular/core/#angular/core/testing.es5.js:671:0 <- src/test.ts:25766:14) [ProxyZone]
at Function.Array.concat.TestBed.configureTestingModule (webpack:///~/#angular/core/#angular/core/testing.es5.js:576:0 <- src/test.ts:25671:22) [ProxyZone]
at Context.<anonymous> (webpack:///src/app/car/reducers/car.reducer.spec.ts:30:12 <- src/test.ts:31440:27) [ProxyZone]
at ProxyZoneSpec.Array.concat.ProxyZoneSpec.onInvoke (webpack:///~/zone.js/dist/proxy.js:79:0 <- src/test.ts:88673:39) [ProxyZone]
at Zone.run (webpack:///~/zone.js/dist/zone.js:125:0 <- src/polyfills.ts:1280:43) [<root> => ProxyZone]
at Context.<anonymous> (webpack:///~/zone.js/dist/mocha-patch.js:88:0 <- src/test.ts:88521:33) [<root>]
Then after the before each fails then we get more errors regarding the property call
Chrome 56.0.2924 (Linux 0.0.0) car reducer "before each" hook for "will return array removes existing values when state is not empty" FAILED
TypeError: Cannot read property 'call' of undefined
at ZoneDelegate.invokeTask (webpack:///~/zone.js/dist/zone.js:398:0 <- src/polyfills.ts:1553:31)
at Zone.runTask (webpack:///~/zone.js/dist/zone.js:165:0 <- src/polyfills.ts:1320:47)
at ZoneTask.invoke (webpack:///~/zone.js/dist/zone.js:460:0 <- src/polyfills.ts:1615:38)
at timer (webpack:///~/zone.js/dist/zone.js:1540:0 <- src/polyfills.ts:2695:29)
Chrome 56.0.2924 (Linux 0.0.0): Executed 45 of 83 (8 FAILED) (0 secs / 0.092 secs)

Answer 1

Multiple references in the frameworks array should be avoided with Karma. Use the plugins instead
This issue around the failing beforeEach was actually a red herring and a side effect of the repeating tests. It appears that the life cycle being executed was:
beforeEach
it('myTest1') pass
it('myTest1') fail
beforeEach
it('myTest2') pass
it('myTest2') fail
If we stop the tests repeating then the beforeEach issue doesn't actually exist. This was digested and demonstrated in a karma issue which turned out to be user (my) error.
To stop the tests repeating we needed to update the frameworks array in the karma.conf.js and reduce the number of framework references and use the plugins instead which were already referenced.
frameworks: ['mocha', 'chai', 'sinon-chai', '#angular/cli'],
becomes
frameworks: ['mocha', '#angular/cli'],
Resolution
Stop the tests repeating and the beforeEach will behave as expected. A working commit can be found here.