Having trouble getting NUnit's Assert.Throws to work properly - unit-testing

I could have sworn that I've used NUnit's Assert.Throws to determine whether or not a particular exception gets thrown from a method, but my memory has failed me before. I read this post here on SO, but it didn't answer my question, as I know the correct syntax, and I don't want to do anything with the exception that gets returned (I don't want to look at the Exception's members, though this could be useful down the road).
I wrote unit tests to prove my lack of understanding in the use of Dictionary, and couldn't get it handle the KeyNotFoundException that gets thrown. Instead of NUnit catching it and passing the test, I get an unhandled KeyNotFoundException error when I run. I verified that I don't have the VS IDE set up to break on thrown .NET exceptions.
I've tried this two ways:
Assert.Throws( typeof(KeyNotFoundException), () => value = prefs["doesn't exist"]);
and
Assert.Throws<KeyNotFoundException>( () => value = prefs["doesn't exist"]);
but both result in an unhandled exception. What am I missing here?
UPDATE seems like others can't reproduce this. Here's a screenshot:

This is an old thread, but try turning off Enable Just My Code in Visual Studio under Tools->Options. With that on, the debugger is trying to be helpful and stops at the last possible point within your code before the exception gets swallowed.
Or, at least that's my understanding of it.
If you turn off Enable Just My Code, the Assert.Throws should work correctly.

The debugger is stating that your exception is not being handled by user code, which is technically true. To demonstrate, I'll use the sample test sgreeve provided
[Test]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
Assert.Throws(typeof(KeyNotFoundException), () => value = test["h"]);
}
When you execute it, you will receive a warning in VisualStudio that the exception is unhandled in user code. If you look at the callstack, you will see something like
[External Code]
CodeTests.DLL!CodeTests.MiscTests.demonstrateThatExceptionThrown.AnonymousMethod()
[External Code]
CodeTests.DLL!CodeTests.MiscTests.demonstrateThatExceptionThrown()
[External Code]
Because you have specified a delegate, the exception is happening within the "AnonymousMethod" that was created. This is being called by the .Net framework. The debugger is stopping because your delegate isn't handling the exception before it gets passed back to the framework. It doesn't care that further up the stack it might be handled in your code (perhaps since there is no way to guarantee that the external code will handle the exception correctly.)
To have VisualStudio see this as a handled exception, use the ExpectedException attribute and remove the delegate, like so:
[Test]
[ExpectedException(typeof(KeyNotFoundException))]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
value = test["h"];
}

Not a direct answer, but I personally prefer to tag my tests with
[ExpectedException(typeof(KeyNotFoundException))]
public Test ShouldDoTheStuff() {
...
}
Does this work for you? I don't actually see anything wrong with your code per se.

EVEN MORE RECENTLY UPDATED ANSWER!
After our conversation in the comments added to this answer, I suspect that the nunitit test runner is the problem here. I don't believe there's anything wrong with your test as I have no problem executing it either using NUnit GUI or the excellent Resharper test runner.
UPDATED ANSWER
After seeing your screen shot, I tried stepping through my test with the debugger and saw exactly the same prompt about the unhandled exception. If I carry on stepping past that error, the test passes when I reach the end of the assertion.
When I run the test in none-debug mode using either the NUnit GUI or the Resharper 4.5 test runner, the test passes as expected every time.
Sorry to ask the obvious question, but what are you executing your test with? i.e. which test runner?
The exact code I've executed is:
using System;
using System.Collections.Generic;
using NUnit.Framework;
namespace ClassLibrary1
{
[TestFixture]
public class DictionaryTest
{
[Test]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
Assert.Throws(typeof(KeyNotFoundException), () => value = test["h"]);
}
}
}

Related

How do I use TestNG SkipException?

How do I use TestNG throw new SkipException() effectively? Does anyone have an example?
I tried throwing this exception at the start of a test method but it blows up the teardown, setup, methods, etc. , and has collateral damage by causing a few (not all) of the subsequent tests to be skipped also, and shows a bunch of garbage on the TestNG HTML report.
I use TestNG to run my unit tests and I already know how to use an option to the #Test annotation to disable a test. I would like my test to show up as "existent" on my report but without counting it in the net result. In other words, it would be nice if there was a #Test annotation option to "skip" a test. This is so that I can mark tests as ignored sortof without having the test disappear from the list of all tests.
Is "SkipException" required to be thrown in #BeforeXXX before the #Test is ran? That might explain the wierdness I am seeing.
Yes, my suspicion was correct. Throwing the exception within #Test doesn't work, and neither did throwing it in #BeforeTest, while I am using parallel by classes. If you do that, the exception will break the test setup and your TestNG report will show exceptions within all of the related #Configuration methods and may even cause subsequent tests to fail without being skipped.
But, when I throw it within #BeforeMethod, it works perfectly. Glad I was able to figure it out. The documentation of the class suggests it will work in any of the #Configuration annotated methods, but something about what I am doing didn't allow me to do that.
#BeforeMethod
public void beforeMethod() {
throw new SkipException("Testing skip.");
}
I'm using TestNG 6.8.1.
I have a few #Test methods from which I throw SkipException, and I don't see any weirdness. It seems to work just as expected.
#Test
public void testAddCategories() throws Exception {
if (SupportedDbType.HSQL.equals(dbType)) {
throw new SkipException("Using HSQL will fail this test. aborting...");
}
...
}
Maven output:
Results :
Tests run: 85, Failures: 0, Errors: 0, Skipped: 2
While using DataProvider empty test using Apache POI create seperate check #BeforeTest we can skip the data base is empty or null in that scenario we can use this skiptest with row check is empty using boolean true check then skipped that expection do not go to entire check its having 1000 input check rather its skip that data provider null...
For skipping test case from #Test annotation option you can use 'enable=false' attribute with #Test annotation as below
#Test(enable=false)
This will skip the test case without running it. but other tests, setup and teardown will run without any issue.

Why do my test functions appear in code coverage? (or how to make them 100%?)

I'm using xUnit to test my C# code and I'm using Visual Studio Premium 2012.
In my solution I have my main project that I'm testing and a 2nd project that contains all of my tests. I'm supposesd to be at 100% code coverage, but there are some functions in my Test project that I cannot get to 100%. Can I just exclude that project from appearing in Code Coverage results?
Or... does anyone now how to get a test function to 100% when you have a test where you are expecting an exception to be thrown? Here are some of the ways I've tried to write a test for a method that should throw an exception and what isn't being covered. MyBusinessLogic has a function named GenerateNameLine that accepts an object of type MyViewModel. if the Name property of MyViewModel is an empty string, it should throw an exception of type RequiredInformationMissingException.
[Fact]
public void TestMethod1()
{
var vm = new MyViewModel();
vm.Name = string.Empty;
Assert.Throws<RequiredInformationMissingException>(delegate { MyBusinessLogic.GenerateNameLine(vm); });
}
This test passes, but code coverage with color highlighting it showing me that MyBusinessLogic.GenerateNameLine(vm); is not getting hit.
I've also tried:
[Fact]
public void TestMethod1
{
bool fRequiredInfoExceptionThrown = false;
var vm = new MyViewModel();
vm.Name = string.Empty;
try
{
MyBusinessLogic.GenerateNameLine(vm);
}
catch (Exception ex)
{
if (ex.GetType() == typeof(RequiredInformationMissingException))
fRequiredInfoExceptionThrown = true;
}
Assert.True(fRequiredInfoExceptionThrown, "RequiredInformationMissingException was not thrown.");
}
This test also passes. But code coverage says the } right before my catch is never hit.
I don't know how to write a test for an exception that gets 100%. I know it doesn't even really matter, but at work 100% code coverage is part of our definition of done, so I don't know what to do here.
The answer is Yes
We provide filters to customize what you want to include/exclude via the .runsettings file. You can filter out pretty much anything that you do not find useful.
The [ExcludeFromCodeCoverage] attribute can also be used in code.
See: http://blogs.msdn.com/b/sudhakan/archive/2012/05/11/customizing-code-coverage-in-visual-studio-11.aspx
Are you seeing the second issue in VS2012RTM+Update1 as well?
I would exclude the tests, but still have an eye on coverage rate for them, because a coverage below 99% would suggest some of them did not run at all.
BTW: 100% is an ideal and cannot be achieved in real life projects. At least the effort to actually reach 100% opposed to something like 90% is disproportionately high. Also exact coverage rates depend on the manner of counting hit lines.

Conflicting results when unit testing MVC controller

I'm writing unit tests (using NUnit & Moq) for my MVC 2 controllers, and am following examples in the Pro ASP.net MVC 2 Framework book by Steven Sanderson (great book, btw). However, I've run into problems, which I think are just due to my lack of understanding of NUnit.
Here's an excerpt, with the irrelevant parts removed:
[Test]
public void Cannot_Save_Invalid_Event()
{
...
repository.Setup(x => x.SaveEvent(evt)).Callback(Assert.Fail);
...
repository.Verify(x => x.SaveEvent(evt));
}
This test is passing for me, although from what I understand, those two statements should directly conflict with each other. The second one wasn't there originally, but I put it in to verify that it was passing for the right reasons.
From what I understand, my repository is set up to fail if "repository.SaveEvent(evt)" is called. However, later in the test, I try to verify that "repository.SaveEvent(evt)" was called. Since it passes, doesn't this mean that it was both called, and not called? Perhaps those statements don't act as I suspect they do.
Can someone explain how these two statements are not opposites, and how they can both exist and the test still pass?
Maybe your tests doesn-t fail beacuse it has a catch-everything block that also hides the assert/verify-exception that is necessary for the test to fail.
Note: the following unittest will allways pass
[Test]
public void HidingAssertionFailure()
{
try {
Assert.AreEqual(0,1); // this should fail
} catch (Exception ex) {
// this will hide the assertion failure
}
}
The reason for this behavior was that it was running "SaveEvent()", however, since the mocked repository didn't define that action, it was throwing an exception in my controller, which my controller was catching.
So, it seems that the callback will only execute if control returns successfully.

Exception handling for Unit Tests in c++

I'm trying to test a c++ code on an Nunit framework but I keep getting the following Exception
System.Runtime.InteropServices.SEHException : External Component has thrown an exception.
which is supposedly perfectly normal (I assume) anyway I wanna ignore it. (i.e. Use ExpectedException) This is my .h file
[Test, Description("Tests if an Entity has been successfully Locked")]
void test_LockOperation();
and the .cpp file
void TestDmObstacles::test_LockOperation()
{
lockVal = DbtoDmObstaclesAdapter::lock( CmnGuid::parseString( L"3B6DB8F8-4BA7-DD11-B6A7-001E8CDE165C" ) );
//When lock is successful the lockVal is 0
Assert::AreEqual(0, lockVal);
}
I wanna use ExpectedException but I don't know how to do it in c++. I tried the try/catch method as well but it didn't work (I just put the Assertion in the catch block)
PS: I can't use another framework it has to be Nunit
EDIT
Here is the try/catch approach I used
void TestDmObstacles::test_LockOperation()
{
try
{
lockVal = DbtoDmObstaclesAdapter::lock( CmnGuid::parseString( L"3B6DB8F8-4BA7-DD11-B6A7-001E8CDE165C" ) );
}
catch (...)
{
//Assert::Fail();
Assert::AreEqual(0, lockVal);
}
}
Is the exception expected, or is the exception acceptable?
If it is expected, then your unit test framework should have some kind of API that allows you to state the expected exception, and to fail the test if it does not occur. A quick trawl through the documentation yields the incantation:
[ExpectedException( "System.ArgumentException" )]
(replace System.ArgumentException with the exception you're expecting.)
If the exception is merely acceptable, then I would say that either your code or your test is broken. A unit test is to test that expected things happen. If there is a result in your test that only may yield a particular result, then you are not testing a consistent view of the unit from test to test. Hence, you're not really testing it.
It might indicate, for example, that your code is leaking an unexpected exception that it should be handling instead.
Your code sample doesn't match what you are trying to achieve : if the exception is expected, than catching it is not supposed to fail the test.
Note that I wouldn't recommend (at all) for the test to catch (...) : any thrown exception will induce the same test result, which I doubt is what you want.

How do I write NUnit unit tests without having to surround them with try catch statements?

At my company we are writing a bunch of unit tests. What we'd like to have done is for the unit tests to execute and whenever one succeeds or fails at the end of the test we can write that somewhere but we don't want to put that logic in every test.
Any idea how we could just write tests without having to surround the content of the test with the try catch logic that we've been using?
I'm guessing you do something like this:
[Test]
public void FailBecauseOfException()
{
try
{
throw new Exception();
}
catch (Exception e)
{
Assert.Fail(e.Message);
}
}
There is no need for this. The tests will fail automatically if they throw an exception. For example, the following test will show up as a failure:
[Test]
public void FailBecauseOfException()
{
throw new Exception();
}
I'm not entirely sure what you are trying to do here. Are you saying you are wrapping it in a try/catch so that you can catch when an exception occurs and log this?
If so, then a better way, probably, is just to get NUnit to write an output file and use this. I haven't used NUnit for about a year, but IIRC you can redirect its output to any file you like using the /out directive.
If there is a reason why you have to log it the way you say, then you'll either have to add your custom code to each test, or have a common "runner" that takes your code (for each test) as an anonymous method and runs it inside a single try..catch. That would prevent you having to repeat the try..catch for every test.
Apologies if I've misunderstood the question.
MSTest has TestCleanup, which runs after every test. In NUnit, the attribute to be used is TearDown (after every test) or TestFixtureTearDown (after all the test are completely). This executes after the end of each test.
If you want something to run just in case a test passes, you could have a member variable shouldRunExtraMethod, which is initialized to false before each test, and is changed to true at the end of the test. And on the TearDown, you only execute it depending on this variable value
If your unit test method covers the scenario in which you expect exceptions to be thrown, use the ExpectedException attribute. There's a post here on SO about using that attribute.
Expect exceptions in nUnit...
NUnit assert statements all have an option to print a message for each test for when it fails.
Although if you'd like to have it write out something somewhere at the end of each test, you can set it up in the teardown of each method. Just set the string to what you want written inside the test itself, and during teardown (which happens after each test) It can do whatever you want with it.
I'm fairly certain teardown occurs even if an exception is thrown. That should do what you're wanting.
The problem you have is that the NUnit Assert.* methods will throw an AssertionException whenever an assert fails - but it does nothing else. So it doesn't look like you can check anything outside of the unit test to verify whether the test failed or not.
The only alternative I can think of is to use AOP (Aspect Oriented Programming) with a tool such as PostSharp. This tool allows you to create aspects that can act on certain events. For example:
public class ExceptionDialogAttribute : OnExceptionAspect
{
public override void OnException(MethodExecutionEventArgs eventArgs)
{
string message = eventArgs.Exception.Message;
Window window = Window.GetWindow((DependencyObject) eventArgs.Instance);
MessageBox.Show(window, message, "Exception");
eventArgs.FlowBehavior = FlowBehavior.Continue;
}
}
This aspect is code which runs whenever an exception is raised:
[ExceptionDialog]
[Test]
public void Test()
{
assert.AreEqual(2, 4);
}
Since the above test will raise an exception, the code in ExceptionDialogAttribute will run. You can get information about the method, such as it's name, so that you can log it into a file.
It's been a long time since I used PostSharp, so it's worth checking out the examples and experimenting with it.