I am using WPToolkitTestFx Unit test framework for the Windows Phone 8 application. I am getting below error when I am executing Assert.Fail() or Assert.IsFalse(true).
A first chance exception of type 'Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException' occurred in Microsoft.VisualStudio.QualityTools.UnitTesting.Phone.DLL
Any solution to the above error.
Here the source code
[TestMethod]
[Asynchronous]
[Description("Test2: Sample asynchronous test")]
public void Test2()
{
// this test executes asynchronously
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
// ... and then fails
Assert.IsFalse(true);
EnqueueTestComplete();
});
}
Thanks,
Ragu
I can see something similar when following the sample code from http://blogs.windows.com/windows_phone/b/wpdev/archive/2012/11/20/windows-phone-toolkit-overview.aspx.
It is normal for the exception to occur, and see:
A first chance exception of type 'Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException' occurred in Microsoft.VisualStudio.QualityTools.UnitTesting.Phone.DLL
Also if the debugger is attached to your device/emulator, the debugger will break so you can find out where the test failure was.
private void Application_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs e)
{
if (Debugger.IsAttached)
{
// An unhandled exception has occurred; break into the debugger
Debugger.Break(); // <- !!! application will stop in debugger here !!!
}
}
However if you continue the project (press F5), or are not running under the debugger, you will see that the application continues to run and doesn't get exited.
That allows you to see your test results.
Question, is it a normal part of your test to fail the assertion? if so, you need to instruct the framework to expect such exception to happen
[TestMethod, Asynchronous]
[Description("Test2: Sample asynchronous test")]
[ExpectedException(typeof(AssertFailedException))]
public void Test2()
{
// this test executes asynchronously
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
// ... and then fails
Assert.IsFalse(true);
//TestComplete();
});
}
Also I notice that you are marking the method as Asynchronous, but you are not using the asynchronous invocation methods of EnqueueDelay(TimeSpan), EnqueueCallback(), EnqueueTestComplete() are the ones that make the method function asynchronous.
Related
Is it possible to test Exceptions with Laravel resource controllers? Every time I try to do the following:
/**
* #expectedException Exception
* #expectedExceptionMessage Just testing this out
*/
public function testMyPost() {
$response = $this->call('POST', '/api/myapi/', array('testing' => 1));
}
I get:
Failed asserting that exception of type "Exception" is thrown.
I've tried this with \Exception and Exception.
In my resource controller I have:
public function store() {
throw new \Exception('Just for testing!');
}
Does anyone has any idea of I can test Exceptions? I've also tried using:
$this->setExpectedException('InvalidArgumentException');
The problem is as hannesvdvreken states; the exception is caught. An easy work-around is to tell Laravels errort/exception handler, that when we are testing, we just want our exceptions thrown.
That could look something like this:
// If we are testing, just throw the exception.
if (App::environment() == 'testing') {
throw $e;
}
For Laravel 5, this should go in the render method in app/Exceptions/Handler.php
For Laravel 4, this should go in app/start/global.php within:
App::error(function(Exception $exception, $code)...
Don't focus on the notation of #expectedException. The problem is the Exception is catched somewhere. Maybe with the default App::error(function(Exception) {... inside the app/start/global.php file.
Or maybe you did a try catch somewhere. Try making a custom Exception that does not get catched by a generic exception catcher that catches everything that's inherited from Exception.
I'm writing many unit tests in VS 2010 with Microsoft Test. In each test class I have many test methods similar to below:
[TestMethod]
public void This_is_a_Test()
{
try
{
// do some test here
// assert
}
catch (Exception ex)
{
// test failed, log error message in my log file and make the test fail
}
finally
{
// do some cleanup with different parameters
}
}
When each test method looks like this I fell it's kind of ugly. But so far I haven't found a good solution to make my test code more clean, especially the cleanup code in the finally block. Could someone here give me some advices on this?
Thanks in advance.
If you really want to handle and log exceptions whilst test execution you can wrap up this standard template in a helper method and use like shown below [*].
But if exceptions is a part of test case this is wrong approach and you should use facilities provided by a test framework, for instance NUnit provides such helpers to test exceptions:
Assert.Throws<ExceptionType>(() => { ... code block... });
Assert.DoesNotThrow(() => { ... code block... });
And to do cleanup special method attributes like [TestCleanup] and [TestInitialize] to do test initialization and cleanup automatically by a test framework.
[*] The idea is to wrap test body in a delegate and pass into the helper which actually perform test execution wrapped in the try/catch block:
// helper
public void ExecuteTest(Action test)
{
try
{
test.Invoke();
}
catch (Exception ex)
{
// test failed, log error message in my log file and make the test fail
}
finally
{
// do some cleanup with different parameters
}
}
[TestMethod]
public void This_is_a_Test_1()
{
Action test = () =>
{
// test case logic
// asserts
};
this.ExecuteTest(test);
}
Clear all try-catch-finally (especially catch is not only unnecessary, but even harmful, you should not catch exception when testing) and do the cleanup in tearDown method (however is it done in MS Test, I would presume there will be something like [TearDownMethod] or [FixtureTearDown] or something like that).
Have you considered using the ExpectedException attribute on the test method?
http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.unittesting.expectedexceptionattribute.aspx
I'm trying to test a c++ code on an Nunit framework but I keep getting the following Exception
System.Runtime.InteropServices.SEHException : External Component has thrown an exception.
which is supposedly perfectly normal (I assume) anyway I wanna ignore it. (i.e. Use ExpectedException) This is my .h file
[Test, Description("Tests if an Entity has been successfully Locked")]
void test_LockOperation();
and the .cpp file
void TestDmObstacles::test_LockOperation()
{
lockVal = DbtoDmObstaclesAdapter::lock( CmnGuid::parseString( L"3B6DB8F8-4BA7-DD11-B6A7-001E8CDE165C" ) );
//When lock is successful the lockVal is 0
Assert::AreEqual(0, lockVal);
}
I wanna use ExpectedException but I don't know how to do it in c++. I tried the try/catch method as well but it didn't work (I just put the Assertion in the catch block)
PS: I can't use another framework it has to be Nunit
EDIT
Here is the try/catch approach I used
void TestDmObstacles::test_LockOperation()
{
try
{
lockVal = DbtoDmObstaclesAdapter::lock( CmnGuid::parseString( L"3B6DB8F8-4BA7-DD11-B6A7-001E8CDE165C" ) );
}
catch (...)
{
//Assert::Fail();
Assert::AreEqual(0, lockVal);
}
}
Is the exception expected, or is the exception acceptable?
If it is expected, then your unit test framework should have some kind of API that allows you to state the expected exception, and to fail the test if it does not occur. A quick trawl through the documentation yields the incantation:
[ExpectedException( "System.ArgumentException" )]
(replace System.ArgumentException with the exception you're expecting.)
If the exception is merely acceptable, then I would say that either your code or your test is broken. A unit test is to test that expected things happen. If there is a result in your test that only may yield a particular result, then you are not testing a consistent view of the unit from test to test. Hence, you're not really testing it.
It might indicate, for example, that your code is leaking an unexpected exception that it should be handling instead.
Your code sample doesn't match what you are trying to achieve : if the exception is expected, than catching it is not supposed to fail the test.
Note that I wouldn't recommend (at all) for the test to catch (...) : any thrown exception will induce the same test result, which I doubt is what you want.
I could have sworn that I've used NUnit's Assert.Throws to determine whether or not a particular exception gets thrown from a method, but my memory has failed me before. I read this post here on SO, but it didn't answer my question, as I know the correct syntax, and I don't want to do anything with the exception that gets returned (I don't want to look at the Exception's members, though this could be useful down the road).
I wrote unit tests to prove my lack of understanding in the use of Dictionary, and couldn't get it handle the KeyNotFoundException that gets thrown. Instead of NUnit catching it and passing the test, I get an unhandled KeyNotFoundException error when I run. I verified that I don't have the VS IDE set up to break on thrown .NET exceptions.
I've tried this two ways:
Assert.Throws( typeof(KeyNotFoundException), () => value = prefs["doesn't exist"]);
and
Assert.Throws<KeyNotFoundException>( () => value = prefs["doesn't exist"]);
but both result in an unhandled exception. What am I missing here?
UPDATE seems like others can't reproduce this. Here's a screenshot:
This is an old thread, but try turning off Enable Just My Code in Visual Studio under Tools->Options. With that on, the debugger is trying to be helpful and stops at the last possible point within your code before the exception gets swallowed.
Or, at least that's my understanding of it.
If you turn off Enable Just My Code, the Assert.Throws should work correctly.
The debugger is stating that your exception is not being handled by user code, which is technically true. To demonstrate, I'll use the sample test sgreeve provided
[Test]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
Assert.Throws(typeof(KeyNotFoundException), () => value = test["h"]);
}
When you execute it, you will receive a warning in VisualStudio that the exception is unhandled in user code. If you look at the callstack, you will see something like
[External Code]
CodeTests.DLL!CodeTests.MiscTests.demonstrateThatExceptionThrown.AnonymousMethod()
[External Code]
CodeTests.DLL!CodeTests.MiscTests.demonstrateThatExceptionThrown()
[External Code]
Because you have specified a delegate, the exception is happening within the "AnonymousMethod" that was created. This is being called by the .Net framework. The debugger is stopping because your delegate isn't handling the exception before it gets passed back to the framework. It doesn't care that further up the stack it might be handled in your code (perhaps since there is no way to guarantee that the external code will handle the exception correctly.)
To have VisualStudio see this as a handled exception, use the ExpectedException attribute and remove the delegate, like so:
[Test]
[ExpectedException(typeof(KeyNotFoundException))]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
value = test["h"];
}
Not a direct answer, but I personally prefer to tag my tests with
[ExpectedException(typeof(KeyNotFoundException))]
public Test ShouldDoTheStuff() {
...
}
Does this work for you? I don't actually see anything wrong with your code per se.
EVEN MORE RECENTLY UPDATED ANSWER!
After our conversation in the comments added to this answer, I suspect that the nunitit test runner is the problem here. I don't believe there's anything wrong with your test as I have no problem executing it either using NUnit GUI or the excellent Resharper test runner.
UPDATED ANSWER
After seeing your screen shot, I tried stepping through my test with the debugger and saw exactly the same prompt about the unhandled exception. If I carry on stepping past that error, the test passes when I reach the end of the assertion.
When I run the test in none-debug mode using either the NUnit GUI or the Resharper 4.5 test runner, the test passes as expected every time.
Sorry to ask the obvious question, but what are you executing your test with? i.e. which test runner?
The exact code I've executed is:
using System;
using System.Collections.Generic;
using NUnit.Framework;
namespace ClassLibrary1
{
[TestFixture]
public class DictionaryTest
{
[Test]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
Assert.Throws(typeof(KeyNotFoundException), () => value = test["h"]);
}
}
}
At my company we are writing a bunch of unit tests. What we'd like to have done is for the unit tests to execute and whenever one succeeds or fails at the end of the test we can write that somewhere but we don't want to put that logic in every test.
Any idea how we could just write tests without having to surround the content of the test with the try catch logic that we've been using?
I'm guessing you do something like this:
[Test]
public void FailBecauseOfException()
{
try
{
throw new Exception();
}
catch (Exception e)
{
Assert.Fail(e.Message);
}
}
There is no need for this. The tests will fail automatically if they throw an exception. For example, the following test will show up as a failure:
[Test]
public void FailBecauseOfException()
{
throw new Exception();
}
I'm not entirely sure what you are trying to do here. Are you saying you are wrapping it in a try/catch so that you can catch when an exception occurs and log this?
If so, then a better way, probably, is just to get NUnit to write an output file and use this. I haven't used NUnit for about a year, but IIRC you can redirect its output to any file you like using the /out directive.
If there is a reason why you have to log it the way you say, then you'll either have to add your custom code to each test, or have a common "runner" that takes your code (for each test) as an anonymous method and runs it inside a single try..catch. That would prevent you having to repeat the try..catch for every test.
Apologies if I've misunderstood the question.
MSTest has TestCleanup, which runs after every test. In NUnit, the attribute to be used is TearDown (after every test) or TestFixtureTearDown (after all the test are completely). This executes after the end of each test.
If you want something to run just in case a test passes, you could have a member variable shouldRunExtraMethod, which is initialized to false before each test, and is changed to true at the end of the test. And on the TearDown, you only execute it depending on this variable value
If your unit test method covers the scenario in which you expect exceptions to be thrown, use the ExpectedException attribute. There's a post here on SO about using that attribute.
Expect exceptions in nUnit...
NUnit assert statements all have an option to print a message for each test for when it fails.
Although if you'd like to have it write out something somewhere at the end of each test, you can set it up in the teardown of each method. Just set the string to what you want written inside the test itself, and during teardown (which happens after each test) It can do whatever you want with it.
I'm fairly certain teardown occurs even if an exception is thrown. That should do what you're wanting.
The problem you have is that the NUnit Assert.* methods will throw an AssertionException whenever an assert fails - but it does nothing else. So it doesn't look like you can check anything outside of the unit test to verify whether the test failed or not.
The only alternative I can think of is to use AOP (Aspect Oriented Programming) with a tool such as PostSharp. This tool allows you to create aspects that can act on certain events. For example:
public class ExceptionDialogAttribute : OnExceptionAspect
{
public override void OnException(MethodExecutionEventArgs eventArgs)
{
string message = eventArgs.Exception.Message;
Window window = Window.GetWindow((DependencyObject) eventArgs.Instance);
MessageBox.Show(window, message, "Exception");
eventArgs.FlowBehavior = FlowBehavior.Continue;
}
}
This aspect is code which runs whenever an exception is raised:
[ExceptionDialog]
[Test]
public void Test()
{
assert.AreEqual(2, 4);
}
Since the above test will raise an exception, the code in ExceptionDialogAttribute will run. You can get information about the method, such as it's name, so that you can log it into a file.
It's been a long time since I used PostSharp, so it's worth checking out the examples and experimenting with it.