Dart unit test giving inconsistent results - unit-testing

I have a fairly simply unit test which has been working without issue for a long time.
It has suddenly started failing.
When I run/debug the unit test in vscode the test succeeds.
When I run the test from the cli it succeeds on windows but not on linux.
dart run test -j1 test/src/functions/ask_test.dart
Which is kind of interesting as there is nothing windows/linux specific.
The test has also been working for a long time on linux.
test('ask.any - throws', () {
final validator = Ask.any([
Ask.fqdn,
Ask.ipAddress(),
Ask.inList(['localhost'])
]);
expect(
() => validator.validate('abc'),
throwsA(predicate<AskValidatorException>((e) =>
e is AskValidatorException && e.message == 'Invalid FQDN.')));
});
The test gives the following error:
Error: Expected: throws satisfies function
Actual: <Closure: () => String>
Which: threw AskValidatorException:<Invalid FQDN.>
stack package:dcli/src/functions/ask.dart 760:7 _AskValidatorAny.validate
test/src/functions/ask_test.dart 65:25 main.<fn>.<fn>
package:test_api expect
test/src/functions/ask_test.dart 64:5 main.<fn>
As you can see validate method is doing exactly what is expected (throwing).
The problem seems to be that the unit test frame work is not seeing it as a match.
This is the validator which is throwing as expected.
class _AskFQDN extends AskValidator {
const _AskFQDN();
#override
String validate(String line) {
final finalLine = line.trim().toLowerCase();
if (!isFQDN(finalLine)) {
throw AskValidatorException(red('Invalid FQDN.'));
}
return finalLine;
}
}
I'm using test 1.17.5 on Dart 2.13.
The code is from the DCli package which you can find here:
https://github.com/bsutton/dcli
The specific unit test is here
https://github.com/bsutton/dcli/blob/master/test/src/functions/ask_test.dart

I am not sure this will solve your problem, but you can make your assertion a bit cleaner if you do it like this:
expect(
() => validator.validate('abc'),
throwsA(isA<AskValidatorException>().having((e) =>
e.message == 'Invalid FQDN.')));

Related

Is a Cache mock called more than once when browser-testing?

I'm trying to cover the following:
I'm using the following test code:
public function test_it_deletes_a_patient()
{
// ...
$cacheKey = vsprintf('%s.%s', [$this->doctorUser->id, 'backoffice.stats.patientsTotalCount']);
Cache::shouldReceive('has')->with($cacheKey)->once()->andReturn(false);
Cache::shouldReceive('increment')->with($cacheKey, -1)->once()->andReturn(true);
$response = $this->json('DELETE', route('patients.destroy', $this->patient), ['confirmation' => 'ELIMINAR']);
// ...
}
That triggers the following controller code:
public function destroy(Patient $patient, Request $request)
{
$this->authorize('delete', $patient);
$confirmation = $request->get('confirmation');
if ($confirmation != 'ELIMINAR') {
return response()->json(['success' => false]);
}
logger()->info("Deleting Patient Profile PATIENT_ID:[{$patient->id}]");
$patient->delete();
$this->updatePatientsCount(-1);
return response()->json(['success' => true]);
}
protected function updatePatientsCount($amount = 1)
{
$key = vsprintf('%s.%s', [auth()->user()->id, 'backoffice.stats.patientsTotalCount']);
if (Cache::has($key)) { // I want to mock for testing this
Cache::increment($key, $amount); // I want to mock for testing this
}
}
After test run I get:
alariva#trinsic:~/fimedi$ t --filter=test_it_deletes_a_patient
PHPUnit 7.3.1 by Sebastian Bergmann and contributors.
F 1 / 1 (100%)
Time: 6.53 seconds, Memory: 26.00MB
There was 1 failure:
1) Tests\Browser\Backoffice\PatientsTest::test_it_deletes_a_patient
Unable to find JSON fragment
["success":true]
within
[{"exception":"Mockery\\Exception\\NoMatchingExpectationException","file":"\/home\/alariva\/fimedi\/vendor\/mockery\/mockery\/library\/Mockery\/ExpectationDirector.php","line":92,"message":"No matching handler found for Mockery_0_Illuminate_Cache_CacheManager::has('2056e535e689ab723b3f44831b488f05f7fb8b90'). Either the method was unexpected or its arguments matched no expected argument list for this method\n\n","trace":[{"class":"App\\Http\\Middleware\\Language","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Barryvdh\\Debugbar\\Middleware\\InjectDebugbar","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Illuminate\\Auth\\Middleware\\Authenticate","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Illuminate\\Cookie\\Middleware\\EncryptCookies","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Il
What I interpret after a couple of tests, is that it looks like once I mock Cache it is being called by some middlewares before reaching the tested block, so since those called methods are not mocked, the test fails because it does not know what to answer for those middleware calls.
Imagine I could successfully mock all the calls before getting to the tested codeblock, I would be able to make it reach. But that's not the way to go over it.
How can I mock Cache and avoid failure due to previous Cache calls that I'm not testing?
EDIT: I realized after getting to a solution that this is a misleading question. My actual need was:
How can I successfully cover those lines?
Sidenote: if I try to disable middlewares ($this->withoutMiddleware();) I get an AccessDeniedHttpException
alariva#trinsic:~/fimedi$ t --filter=test_it_deletes_a_patient
PHPUnit 7.3.1 by Sebastian Bergmann and contributors.
F 1 / 1 (100%)
Time: 12.95 seconds, Memory: 24.00MB
There was 1 failure:
1) Tests\Browser\Backoffice\PatientsTest::test_it_deletes_a_patient
Unable to find JSON fragment
["success":true]
within
[{"exception":"Symfony\\Component\\HttpKernel\\Exception\\AccessDeniedHttpException","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Exceptions\/Handler.php","line":201,"message":"This action is unauthorized.","trace":[{"class":"App\\Exceptions\\Handler","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Routing\/Pipeline.php","function":"render","line":83,"type":"->"},{"class":"Illuminate\\Foundation\\Exceptions\\Handler","file":"\/home\/alariva\/fimedi\/app\/Exceptions\/Handler.php","function":"render","line":65,"type":"->"},{"class":"Illuminate\\Foundation\\Exceptions\\Handler","file":
Maybe I can cherry-pick middlewares to disable?
I managed to cover the controller's method by encapsulating the custom Cache operation into a macro, so as to get the benefits of spliting into code units.
I moved my code into a macro (in the boot() of a service provider):
Cache::macro('incrementExisting', function($key, $amount) {
if (Cache::has($key)) {
Cache::increment($key, $amount);
}
return $this;
});
I refactored to use the macro
protected function updatePatientsCount($amount = 1)
{
$key = vsprintf('%s.%s', [auth()->user()->id, 'backoffice.stats.patientsTotalCount']);
Cache::incrementExisting($key, $amount);
}
I could get the desired coverage while I can still test the refactored code with unit testing.
Update I
Regarding the concern of handling many calls that are not mocked, I just learned from Adam Wathan that there exists shouldIgnoreMissing() and that would allow to use the Mocking approach for this case.
Update II
Write your tests first. When doing so it gets easier to avoid hard-to-test code.

Jest expect().toEqual() not throwing error

I expected the following sample test to fail if I use toEqual(), but it passes:
it('sample test to check toEqual and toBe', () => {
const expectedAction = {test: '123', value: undefined};
const actualAction = {test: '123'};
expect(expectedAction).toEqual(actualAction);
})
The same test fails if I use .toBe(). However, if I fix the test case like this:
it('sample test to check toEqual and toBe', () => {
const expectedAction = {test: '123', value: '123'};
const actualAction = {test: '123', '123'};
expect(expectedAction).toBe(actualAction);
});
it again fails saying that "Compared values have no visual difference".
Is the behaviour correct? How to modify my test case so that it fails correctly.
If you really need it to be considered as different, one option would be to use Jest snapshot test, like:
expect(expectedAction).toMatchSnapshot();
The first time, it will create a snapshot file and print the javascript object to it, when you run the test again, it will compare it to the snapshot.
Most of the times, the snapshots are used to compare the component rendered tree, but you can use it for any javascript object.
The first one passes cause toEqual compares every element of your objects with each other, so in your case
expectedAction.test === actualAction.test
as both are '123'
but the same is true for
expectedAction.value === actualAction.value
as both are undefined
The second test fails cause toBe uses === to compare the two objects which will of cause fail as they are not the same instances. The only way that an object will passes toBe would be to use itself for comparison:
expect(expectedAction).toBe(expectedAction)

RxJs 5 ascii marbles output for failing test

I am learning RxJS 5 (release candidate). I am using the ASCII marble diagrams in unit tests.
For a failing test, is there a built in function to show the actual ASCII marbles string instead of the deep equal failure result? It seems like if I have to understand and write an expected ASCII marble string that it would be useful for a failing test to show the actual ASCII marble string. It would assist in troubleshooting.
I am hoping that there is a built in function that is effectively the reverse of parseMarbles(). I have not found anything yet when searching through the documentation and source code.
Below is an example taken from Writing marble tests - anatomy of a test that I modified to get a failing test. It gives the following output:
AssertionError: expected [ Array(5) ] to deeply equal [ Array(4) ]Error
I would rather see something like:
AssertionError: expected "'---(be)----c-f-----|'"
to equal "'---(be)----c-------|'"
http://jsbin.com/hipepew/edit?js,output
mocha.setup('bdd');
const expect = chai.expect;
describe('RXJS 5 Marbles', () => {
it("basic anatomy of a failing test", () => {
const rxTestScheduler: TestScheduler = new Rx.TestScheduler(function (actual, expected) {
expect(actual).to.deep.equal(expected);
});
const e1 = rxTestScheduler.createHotObservable<string>('----a--^--b-------c--|');
const e2 = rxTestScheduler.createHotObservable<string>('---d-^--e---------f-----|');
const expectedFail = '---(be)----c-------|';//missing f at frame 130
rxTestScheduler.expectObservable(e1.merge(e2)).toBe(expectedFail);
rxTestScheduler.flush();
});
});
mocha.run();

Unit testing grails controllers that use declarative exception handling

Let's say I have the following code:
class SomeController {
def fooService
def controllerMethod() {
def bar = fooService.doSomething()
// render bar to user - success case
}
def fooExceptionHandler(FooException e) {
// log, render error page, etc...
}
}
Based on grails' new declarative controller exception handling mechanism, if fooService.doSomething() throws an exception, grails will call fooExceptionHandler for me. Great.
Now when I unit test this method (test class uses the #TestFor(SomeController) annotation), this will fail saying that we expected a FooException but got nothing.
#Test(expected=FooException)
def doSomethingThrowsFooException() {
// override default service behavior, trigger a FooException
controller.fooService = [ doSomething: { throw new FooException() }]
controller.controllerMethod()
}
However, this works:
#Test
def doSomethingThrowsFooException() {
// override default service behavior, trigger a FooException
controller.fooService = [ doSomething: { throw new FooException() }]
controller.controllerMethod()
assert response.json == false
}
So the only way to test this method is by asserting that the response was what was expected but due to the declarative exception handling, this logic is now somewhere else (tested in isolation), not in the unit of code I am testing. Shouldn't my unit test only verify that the exception was propagated out of the controller method?
I would test the fooService integration with your controller in an integration test if possible, but I think that's only really testable sensibly in a functional test. You're really testing Grails - you're verifying that documented behavior occurs in your app.
If you add some code to throw an exception and then extra code to catch it and route it to the handler, you're just mocking out the stuff that Grails provides. Tests of that just test your mocking code, but have little to do with how well your code will work in production.
The unit test opportunity here is inside fooExceptionHandler. Test that given an exception, you do the right thing with it.

Working with GrailsUnitTestCase and Spock Specification in the same project

I have existing UnitTests written extending grails' GrailsUnitTestCase. I wrote new Unit Tests using Spock framework. Running individual Spock Spec with following command is fine.
prayag#prayag:~/gccount$ grails test-app spock: ChronicSpec
But running existing UnitTests throws compilation error in Spock Tests.
prayag#prayag:~/gccount$ grails test-app unit: ChronicUnitTest.testDefaultChronic
| Error Compilation error compiling [unit] tests: startup failed:
/home/prayag/gccount/test/unit/com/zazzercode/chronic/ChronicSpec.groovy: 45: unexpected token: } # line 45, column 5.
} //end of response process
^
1 error
ChronicSpec.groovy is as below
class ChronicSpec extends Specification {
final public String CHRONIC_DEFAULT_METRIC = ReportType.CHRONIC_REPORT + Constants.COLON + ChronicMetrics.DEFAULT
final public String INDEX_NAME = "gccount"
ChronicService chronicService
void "when response is processed json should be created"() {
given:
def contentBuilder = XContentFactory.jsonBuilder().startObject()
chronicService = new ChronicService()
//other code goes here
when:
listener.writeJson(records, contentBuilder, "chronic")
then:
//assertion goes here
} //end of response process => this is where exception is thrown
}
Tools 'm using
grails : 2.2.3
spock : 0.7
I am surprised this actually compiles, you cannot leave an empty "then:" block.
You need at least one assertion. As suggested by Sergio Michels, add an assertion and it should fix it.