I'm writing many unit tests in VS 2010 with Microsoft Test. In each test class I have many test methods similar to below:
[TestMethod]
public void This_is_a_Test()
{
try
{
// do some test here
// assert
}
catch (Exception ex)
{
// test failed, log error message in my log file and make the test fail
}
finally
{
// do some cleanup with different parameters
}
}
When each test method looks like this I fell it's kind of ugly. But so far I haven't found a good solution to make my test code more clean, especially the cleanup code in the finally block. Could someone here give me some advices on this?
Thanks in advance.
If you really want to handle and log exceptions whilst test execution you can wrap up this standard template in a helper method and use like shown below [*].
But if exceptions is a part of test case this is wrong approach and you should use facilities provided by a test framework, for instance NUnit provides such helpers to test exceptions:
Assert.Throws<ExceptionType>(() => { ... code block... });
Assert.DoesNotThrow(() => { ... code block... });
And to do cleanup special method attributes like [TestCleanup] and [TestInitialize] to do test initialization and cleanup automatically by a test framework.
[*] The idea is to wrap test body in a delegate and pass into the helper which actually perform test execution wrapped in the try/catch block:
// helper
public void ExecuteTest(Action test)
{
try
{
test.Invoke();
}
catch (Exception ex)
{
// test failed, log error message in my log file and make the test fail
}
finally
{
// do some cleanup with different parameters
}
}
[TestMethod]
public void This_is_a_Test_1()
{
Action test = () =>
{
// test case logic
// asserts
};
this.ExecuteTest(test);
}
Clear all try-catch-finally (especially catch is not only unnecessary, but even harmful, you should not catch exception when testing) and do the cleanup in tearDown method (however is it done in MS Test, I would presume there will be something like [TearDownMethod] or [FixtureTearDown] or something like that).
Have you considered using the ExpectedException attribute on the test method?
http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.unittesting.expectedexceptionattribute.aspx
Related
The idea is to execute a common initialization for all unit tests in the suite, I mean, like executing a setupAll() before any test (that are written already) . Then I'd have an external component like setupInitializers() that is a global function or something that I could put on every unit test file easily and runs before each test.
void main() {
setupInitializers();
...
...
// rest of the unit tests
}
Is it clear the idea?
Turns out that setUpAll actually exists.
It requires a dynamic Function as a parameter, like the example below:
void main() {
setUpAll(setupInitializers);
test('your test here', () {
// ...
});
}
void setupInitializers() {
// your initialization here
}
For more information you can go over the official documentation.
I use xUnit and FluentAssertions to write my unit tests and I am stuck at the following problem. As I have not implemented the catch (in GetCountriesAsync) of the WebException yet, I throw a new NotImplementedException in this place.
This code is the only way I made the test actually work as expected. I added the native xUnit implementation either, due to FluentAssertions is just syntactic sugar.
[Fact]
public async Task GetCountriesAsyncThrowsExceptionWithoutInternetConnection()
{
// Arrange
Helpers.Disconnect(); // simulates network disconnect
var provider = new CountryProvider();
try
{
// Act
var countries = await provider.GetCountriesAsync();
}
catch (Exception e)
{
// Assert FluentAssertions
e.Should().BeOfType<NotImplementedException>();
// Assert XUnit
Assert.IsType<NotImplementedException>(e);
}
}
Though I found this implementation a lot nicer, it just doesn't work.
[Fact]
public async Task GetCountriesAsyncThrowsExceptionWithoutInternetConnection3()
{
// Arrange
Helpers.Disconnect(); // simulates network disconnect
var provider = new CountryProvider();
// Act / Assert FluentAssertions
provider.Invoking(async p => await p.GetCountriesAsync())
.ShouldThrow<NotImplementedException>();
// Act / Assert XUnit
Assert.Throws<NotImplementedException>(async () => await provider.GetCountriesAsync());
}
As VS2012/ReSharper already suggests to remove the redundant async keyword of the test method, I replaced async Task with void and the test still behaves the same, so I suspect the async Actions cannot be awaited, they're rather fired and forgotten.
Is there a way to implement this properly with xUnit/FluentAssertions? I think I have to go with my first implementation as I can't see any functionality like InvokingAsync().
Actually, FA 2.0 has specific support for handling asynchronous exceptions. Just look at the unit tests in AsyncFunctionExceptionAssertionSpecs. for various examples.
Regarding FluentAssertions, I've added the following to my code:
using System;
using System.Threading.Tasks;
namespace FluentAssertions
{
public static class FluentInvocationAssertionExtensions
{
public static Func<Task> Awaiting<T>(this T subject, Func<T, Task> action)
{
return () => action(subject);
}
}
}
and now you can do:
_testee.Awaiting(async x => await x.Wait<Foo>(TimeSpan.Zero))
.ShouldThrow<BarException>();
whereas _teste.Wait<T> returns a Task<T>.
Also the naming of the method Awaiting make sense, because pure invocation of the method will not result in the exception being caught by the caller, you do need to use await to do so.
I'm using groovy to write unit tests for my java project (I'd advise all to try it out)
One of my unit tests is testing that the tested code (in java) catches an exception of a specific type and rethrows an app exception:
java code (under test):
try
{
dao.save(obj)
}
catch(DataIntegrityException dupex)
{
....
throw new AppException("duplicate name");
}
The groovy test code mocks(proxies) the dao call using groovy's map of closures
[ save:
{ obj->
...
throw new DataIntegrityException("duplicate"); //DataIntegrityException is a runtime exception (unchecked)
}
] as DAO;
Pretty straightforward. However, when executing the test class, the actual exception thrown is groovy's runtime.Invoker.InvocationException which wraps the DataIntegrity exception. Obviously, the java code knows nothing about groovy and its exceptions, so the catch block is not executed.
I'm using Netbeans 7 to actually execute the test and it just uses groovyc to compile the groovy file into a java class, and so it's just being executed as a normal JUnit test.
Anyone know how to overcome this? couldn't find it listed in groovy's bugs, but sure seems like one.
Just using this for a little more space, not yet an answer. I can get the following (minimal) setup to work:
Dao.java
public interface Dao { void save(Object obj) throws DataIntegrityException; }
DataIntegrityException.java
public class DataIntegrityException extends RuntimeException {
public DataIntegrityException(String str) { super(str); }
}
Test.java
public class Test {
public void doIt(Dao dao) {
try {
dao.save(new Object());
} catch (DataIntegrityException e) {
System.out.println("Caught DataIntegrityException")
}
}
}
test_case.groovy
d = [save: { throw new DataIntegrityException('duplicate') }] as Dao
new Test().doIt(d)
This works, i.e., it prints "Caught DataIntegrityException" to the console, using Java 6 and Groovy 1.8 (which is what I have on my machine. At the very least, the issue must lie somewhere outside of the code you've posted here.
Personally, I'd think you'd need some step-through debugging. It seems likely that the exception is being thrown from elsewhere in the code (perhaps in whatever is contained in the ellipses in your version prior to throw new DataIntegrityException()).
I'm writing unit tests (using NUnit & Moq) for my MVC 2 controllers, and am following examples in the Pro ASP.net MVC 2 Framework book by Steven Sanderson (great book, btw). However, I've run into problems, which I think are just due to my lack of understanding of NUnit.
Here's an excerpt, with the irrelevant parts removed:
[Test]
public void Cannot_Save_Invalid_Event()
{
...
repository.Setup(x => x.SaveEvent(evt)).Callback(Assert.Fail);
...
repository.Verify(x => x.SaveEvent(evt));
}
This test is passing for me, although from what I understand, those two statements should directly conflict with each other. The second one wasn't there originally, but I put it in to verify that it was passing for the right reasons.
From what I understand, my repository is set up to fail if "repository.SaveEvent(evt)" is called. However, later in the test, I try to verify that "repository.SaveEvent(evt)" was called. Since it passes, doesn't this mean that it was both called, and not called? Perhaps those statements don't act as I suspect they do.
Can someone explain how these two statements are not opposites, and how they can both exist and the test still pass?
Maybe your tests doesn-t fail beacuse it has a catch-everything block that also hides the assert/verify-exception that is necessary for the test to fail.
Note: the following unittest will allways pass
[Test]
public void HidingAssertionFailure()
{
try {
Assert.AreEqual(0,1); // this should fail
} catch (Exception ex) {
// this will hide the assertion failure
}
}
The reason for this behavior was that it was running "SaveEvent()", however, since the mocked repository didn't define that action, it was throwing an exception in my controller, which my controller was catching.
So, it seems that the callback will only execute if control returns successfully.
At my company we are writing a bunch of unit tests. What we'd like to have done is for the unit tests to execute and whenever one succeeds or fails at the end of the test we can write that somewhere but we don't want to put that logic in every test.
Any idea how we could just write tests without having to surround the content of the test with the try catch logic that we've been using?
I'm guessing you do something like this:
[Test]
public void FailBecauseOfException()
{
try
{
throw new Exception();
}
catch (Exception e)
{
Assert.Fail(e.Message);
}
}
There is no need for this. The tests will fail automatically if they throw an exception. For example, the following test will show up as a failure:
[Test]
public void FailBecauseOfException()
{
throw new Exception();
}
I'm not entirely sure what you are trying to do here. Are you saying you are wrapping it in a try/catch so that you can catch when an exception occurs and log this?
If so, then a better way, probably, is just to get NUnit to write an output file and use this. I haven't used NUnit for about a year, but IIRC you can redirect its output to any file you like using the /out directive.
If there is a reason why you have to log it the way you say, then you'll either have to add your custom code to each test, or have a common "runner" that takes your code (for each test) as an anonymous method and runs it inside a single try..catch. That would prevent you having to repeat the try..catch for every test.
Apologies if I've misunderstood the question.
MSTest has TestCleanup, which runs after every test. In NUnit, the attribute to be used is TearDown (after every test) or TestFixtureTearDown (after all the test are completely). This executes after the end of each test.
If you want something to run just in case a test passes, you could have a member variable shouldRunExtraMethod, which is initialized to false before each test, and is changed to true at the end of the test. And on the TearDown, you only execute it depending on this variable value
If your unit test method covers the scenario in which you expect exceptions to be thrown, use the ExpectedException attribute. There's a post here on SO about using that attribute.
Expect exceptions in nUnit...
NUnit assert statements all have an option to print a message for each test for when it fails.
Although if you'd like to have it write out something somewhere at the end of each test, you can set it up in the teardown of each method. Just set the string to what you want written inside the test itself, and during teardown (which happens after each test) It can do whatever you want with it.
I'm fairly certain teardown occurs even if an exception is thrown. That should do what you're wanting.
The problem you have is that the NUnit Assert.* methods will throw an AssertionException whenever an assert fails - but it does nothing else. So it doesn't look like you can check anything outside of the unit test to verify whether the test failed or not.
The only alternative I can think of is to use AOP (Aspect Oriented Programming) with a tool such as PostSharp. This tool allows you to create aspects that can act on certain events. For example:
public class ExceptionDialogAttribute : OnExceptionAspect
{
public override void OnException(MethodExecutionEventArgs eventArgs)
{
string message = eventArgs.Exception.Message;
Window window = Window.GetWindow((DependencyObject) eventArgs.Instance);
MessageBox.Show(window, message, "Exception");
eventArgs.FlowBehavior = FlowBehavior.Continue;
}
}
This aspect is code which runs whenever an exception is raised:
[ExceptionDialog]
[Test]
public void Test()
{
assert.AreEqual(2, 4);
}
Since the above test will raise an exception, the code in ExceptionDialogAttribute will run. You can get information about the method, such as it's name, so that you can log it into a file.
It's been a long time since I used PostSharp, so it's worth checking out the examples and experimenting with it.