When running this test:
const file = "exp.txt";
Deno.test("handle rejected promise", async (t) => {
try {
await Deno.stat(file);
} catch (e) {
if (e instanceof Deno.errors.NotFound) {
await Deno.writeTextFile(file, "some text");
} else {
throw e;
}
} finally {
await Deno.remove(file);
}
});
When running it like so:
deno test --allow-read --allow-write exp.test.ts
the test succeeds.
However, when running it like so (adding the trace-ops option):
deno test --allow-read --allow-write --trace-ops exp.test.ts
it fails with:
./exp.test.ts (uncaught error)
error: (in promise) NotFound: The system cannot find the file specified. (os error 2), stat 'exp.txt'
This error was not caught from a test and caused the test runner to fail on the referenced module.
It most likely originated from a dangling promise, event/timeout handler or top-level code.
I have more involved code doing essentially the above failing even without --trace-ops. It appears to be more of a race condition or some timeout.
This is deno 1.28.3 and I've tried with several 1.2x versions with flaky results. However, it never bit me before running 1.28.3.
Note this way of checking a file's existence is suggested by the deno docs e.g. here
This turned out to be a bug and after submitting an issue it was fixed and should be included in a new release of Deno (1.28.3+).
Related
I'm in the process of returning to ReSharper recently, using trial of the newest version - 2022.2.3. I've got quite surprised when one of my nUnit tests failed in a weird way, when run by Resharper's built in Unit Test runner. Something that has never happened to me with a Test Explorer.
As long as the Asserts pass, it's all fine - green, all tests are listed. However, when the assert fails, it says One or more child tests had errors. Exception doesn't have a stacktrace
Not only there is no mention of actual values that weren't correct, but the whole failing test seems to be gone!
This happens only when I use the 'modern' approach with Assert.That. So
Assert.That(httpContext.Response.StatusCode, Is.EqualTo(200));
is causing issues, meanwhile, the more classic:
Assert.AreEqual(200, httpContext.Response.StatusCode);
works as expected. Is that something that is a known bug, or maybe some attributes are required? JetBrains claims they have full support of nUnit out of the box, so that is a bit surprising.
NOTE: the tests methods are async, awaiting result and returning Tasks, beside this nothing unusual.
EDIT: The test code is as follows, ApiKeyMiddleware is any middleware that returns response with 200 here.
[TestFixture]
public class ApiKeyMiddlewareTests
{
[Test]
public async Task Invoke_ActiveKey_Authorized()
{
var httpContext = new DefaultHttpContext();
httpContext.Request.Headers.Add("XXXXX", "xxxx");
var configuration = Options.Create(new AccessConfiguration { ActiveApiKeys = new List<string> { "xxxx" } });
var middleware = new ApiKeyMiddleware(GetEmptyRequest(), configuration);
await middleware.Invoke(httpContext);
Assert.That(httpContext.Response.StatusCode, Is.EqualTo(200)); //change to anything else than 200 and it fails + vanishes
}
}
I am new to kotlin multiplatform library.
I wanted to make a simple HTTP get request and test if it works.
here is what I have so far.
this is in the commonMain package
import io.ktor.client.*
import io.ktor.client.request.*
object HttpCall {
private val client: HttpClient = HttpClient()
suspend fun request(url: String): String = client.get(url)
}
and here is my attempt to test
#Test
fun should_make_http_call() {
GlobalScope.launch {
val response = HttpCall.request("https://stackoverflow.com/")
println("Response: ->$response")
assertTrue { response.contains("Stack Overflow - Where Developers Learn") }
assertTrue { response.contains("text that does not exist on stackoverflow") }
}
Now, this should fail because of the second assert but it doesn't.
no matter what I do the test always passes.
and printing the response does not work either
what am I doing wrong here?
The test function will run in a single thread, and if the function ends without failing, the test passes. GlobalScope.launch starts an operation in a different thread. The main test thread will finish before the network calls get a chance to run.
You should be calling this with something like runBlocking, but testing coroutines in general, and ktor specifically, on Kotlin native, is not easy because there's no easy way to have the suspended function continue on your current thread.
I will not use the GlobalScope or the runBlocking because they are not really made for Unit Test. Instead, I will use runTest.
Here are the steps:
Check your build.gradle and make sure you do have under commontTest the lib 'kotlinx-coroutines-test' set
val commonTest by getting {
dependencies {
implementation(kotlin("test"))
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:${Version.kotlinCoroutines}")
...
}
}
Then into your directory commonTest, create a file and run
#Test
fun should_make_http_call() = runTest {
val response = HttpCall.request("https://stackoverflow.com/")
println("Response: ->$response")
assertTrue { response.contains("Stack Overflow - Where Developers Learn") }
assertTrue { response.contains("text that does not exist on stackoverflow") }
}
Extra:
runTest does not handle exceptions very well, so if you are interested in making to catch any exceptions if happens. Change runTest for runReliableTest, You can get the code from this link https://github.com/Kotlin/kotlinx.coroutines/issues/1205#issuecomment-1238261240
I've found a lot of this sort of thing when refactoring our Jest test suites:
it('calls the API and throws an error', async () => {
expect.assertions(2);
try {
await login('email', 'password');
} catch (error) {
expect(error.name).toEqual('Unauthorized');
expect(error.status).toEqual(401);
}
});
I believe the expect.assertions(2) line is redundant here, and can safely be removed, because we already await the async call to login().
Am I correct, or have I misunderstood how expect.assertions works?
expect.assertions is important when testing the error scenarios of asynchronous code, and is not redundant.
If you remove expect.assertions from your example you can't be confident that login did in fact throw the error.
it('calls the API and throws an error', async () => {
try {
await login('email', 'password');
} catch (error) {
expect(error.name).toEqual('Unauthorized');
expect(error.status).toEqual(401);
}
});
Let's say someone changes the behavior of login to throw an error based on some other logic, or someone has affected the mock for this test which no longer causes login to throw. The assertions in the catch block won't run but the test will still pass.
Using expect.assertions at the start of the test ensures that if the assertions inside the catch don't run, we get a failure.
This is from Jest documentation:
Expect.assertions(number) verifies that a certain number of assertions
are called during a test. This is often useful when testing
asynchronous code, in order to make sure that assertions in a callback
actually got called.
So to put in other words, expect.assertions makes sure that the n number of assertions are made by the end of the test.
It's good to use it especially when writing a new tests, so one can easily check that correct assertions are made during the test. Async tests often pass because the intended assertions were not made before the test-runner (Jest,Mocha etc.) thought the test was finished.
I think we are missing the obvious here.
expect.assertions(3) is simply saying ...
I expected 3 expect statements to be called before the test times out. e.g.
expect(actual1).toEqual(expected1);
expect(actual2).toEqual(expected2);
expect(actual3).toEqual(expected3);
This timing out business is the reason to use expect.assertions. It would be silly to use it in a purely synchronous test. At least one of the expect statements would be found in a subscribe block (or other async block) within the spec file.
To ensure that the assertions in the catch block of an async/await test are adequately tested, expect.assertions(n) must be declared as shown in your code snippet. Such declaration is unnecessary for async/await tests without the catch block.
It seems quite unintuitive but it is simply the way it is. Perhaps, for certain reasons well deep within the javascript runtime, the test environment can detect when an await'ed' promise successfully resolved but cannot detect same for await'ed' promises that failed to resolve. The creators of the test environment would likely know verbatim why such is the case.
I have to admit that apart from error testing, I find it challenging to see a real use for expect.assertions. The above snippet can be changed to the following with the same guarantee but I think it reads more naturally and doesn't require me to count how many time I call expect. This is especially error-prone if a test if complex:
it('calls the API and throws an error', async () => {
try {
await login('email', 'password');
fail('must throw')
} catch (error) {
expect(error.name).toEqual('Unauthorized');
expect(error.status).toEqual(401);
}
});
I need to test a directive, and it should throw an exception. How can I test that the exception was thrown, in jasmine?
The directives link function:
link: function() {
if(something) {
throw new TypeError('Error message');
}
}
I have not yet successfully implemented a test that actually catches the error and reports that the test was successful.
This is how I do it:
describe("myDirective", function() {
it("should throw an error", inject(function ($compile, $rootScope) {
function errorFunctionWrapper()
{
$compile(angular.element("<div my-directive></div>"))($rootScope);
}
expect(errorFunctionWrapper).toThrow();
}));
});
it("should throw an error", inject(function ($compile, $rootScope) {
expect(function () {
$compile(angular.element("<directive-name></directive-name>"))($rootScope.$new());
}).toThrow();
}));
EDIT: this seems to be fixed now. Tested with Angular 1.6.4.
In Angular 1.6, I'm been unable to catch errors thrown during the $compile phase. While not as elegant, I can still check against the $exceptionHandler.errors array (source):
it('throws an error', function() {
$compile(angular.element('<directive-name></directive-name>'))($rootScope.$new());
expect($exceptionHandler.errors.length).toBeGreaterThan(0);
});
Better yet, provide it with the exact error message(s):
expect($exceptionHandler.errors).toEqual(['first error', 'second error'])
exception handling in angular testing is better done with native angular service
https://docs.angularjs.org/api/ng/service/$exceptionHandler
this gives better handle on the thrown exceptions and provide a better native angular way to handle exception globally. say if at some point in time you can change the exception handling strategy of you application in a single place.
in testing this when used along with $exceptionHandlerProvider
https://docs.angularjs.org/api/ngMock/provider/$exceptionHandlerProvider
gives you a better handle on the generated exception and write specific tests.
for unit tests it is not a standard way in angular to validate an exception using .toThrow(); method of jasmine.
I am using WPToolkitTestFx Unit test framework for the Windows Phone 8 application. I am getting below error when I am executing Assert.Fail() or Assert.IsFalse(true).
A first chance exception of type 'Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException' occurred in Microsoft.VisualStudio.QualityTools.UnitTesting.Phone.DLL
Any solution to the above error.
Here the source code
[TestMethod]
[Asynchronous]
[Description("Test2: Sample asynchronous test")]
public void Test2()
{
// this test executes asynchronously
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
// ... and then fails
Assert.IsFalse(true);
EnqueueTestComplete();
});
}
Thanks,
Ragu
I can see something similar when following the sample code from http://blogs.windows.com/windows_phone/b/wpdev/archive/2012/11/20/windows-phone-toolkit-overview.aspx.
It is normal for the exception to occur, and see:
A first chance exception of type 'Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException' occurred in Microsoft.VisualStudio.QualityTools.UnitTesting.Phone.DLL
Also if the debugger is attached to your device/emulator, the debugger will break so you can find out where the test failure was.
private void Application_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs e)
{
if (Debugger.IsAttached)
{
// An unhandled exception has occurred; break into the debugger
Debugger.Break(); // <- !!! application will stop in debugger here !!!
}
}
However if you continue the project (press F5), or are not running under the debugger, you will see that the application continues to run and doesn't get exited.
That allows you to see your test results.
Question, is it a normal part of your test to fail the assertion? if so, you need to instruct the framework to expect such exception to happen
[TestMethod, Asynchronous]
[Description("Test2: Sample asynchronous test")]
[ExpectedException(typeof(AssertFailedException))]
public void Test2()
{
// this test executes asynchronously
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
// ... and then fails
Assert.IsFalse(true);
//TestComplete();
});
}
Also I notice that you are marking the method as Asynchronous, but you are not using the asynchronous invocation methods of EnqueueDelay(TimeSpan), EnqueueCallback(), EnqueueTestComplete() are the ones that make the method function asynchronous.