This is my Unit Test:
[TestCategory("Repo")]
[TestMethod]
[ExpectedException(typeof(System.ServiceModel.FaultException))]
public void RepoUnitTest_ExpectError()
{
var repo = new CreateClientRepository(ClientServiceWrapper);
IClientObject input = new ClientObject
{
ClientId = 0,
ClientName = null
};
repo.CreateClient(input);
}
I am providing invalid input to my client repository which in turn calls a third party client service, and I expect the client service to throw an error. And, client service throws an exception too, but not the way I expect it. I expect it "System.ServiceModel.FaultException", but it give me this:
Test method RepoUnitTest_ExpectError threw exception
System.ServiceModel.FaultException`1[[System.ServiceModel.ExceptionDetail, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]],
but exception System.ServiceModel.FaultException was expected.
Not sure what do I put in "ExpectedException" so this Unit Test passes with correctly expected exception.
Guess I should have waited few more minutes before posting this question, but I am glad did, now I can share the knowledge with whoever stumbles upon similar issue.
The expected exception should be written this way (for this particular case):
[ExpectedException(typeof(System.ServiceModel.FaultException<
System.ServiceModel.ExceptionDetail>))]
Related
I'm in the process of returning to ReSharper recently, using trial of the newest version - 2022.2.3. I've got quite surprised when one of my nUnit tests failed in a weird way, when run by Resharper's built in Unit Test runner. Something that has never happened to me with a Test Explorer.
As long as the Asserts pass, it's all fine - green, all tests are listed. However, when the assert fails, it says One or more child tests had errors. Exception doesn't have a stacktrace
Not only there is no mention of actual values that weren't correct, but the whole failing test seems to be gone!
This happens only when I use the 'modern' approach with Assert.That. So
Assert.That(httpContext.Response.StatusCode, Is.EqualTo(200));
is causing issues, meanwhile, the more classic:
Assert.AreEqual(200, httpContext.Response.StatusCode);
works as expected. Is that something that is a known bug, or maybe some attributes are required? JetBrains claims they have full support of nUnit out of the box, so that is a bit surprising.
NOTE: the tests methods are async, awaiting result and returning Tasks, beside this nothing unusual.
EDIT: The test code is as follows, ApiKeyMiddleware is any middleware that returns response with 200 here.
[TestFixture]
public class ApiKeyMiddlewareTests
{
[Test]
public async Task Invoke_ActiveKey_Authorized()
{
var httpContext = new DefaultHttpContext();
httpContext.Request.Headers.Add("XXXXX", "xxxx");
var configuration = Options.Create(new AccessConfiguration { ActiveApiKeys = new List<string> { "xxxx" } });
var middleware = new ApiKeyMiddleware(GetEmptyRequest(), configuration);
await middleware.Invoke(httpContext);
Assert.That(httpContext.Response.StatusCode, Is.EqualTo(200)); //change to anything else than 200 and it fails + vanishes
}
}
I have a web service with an operation that looks like
public Result checkout(String id) throws LockException;
implemented as:
#Transactional
public Result checkout(String id) throws LockException {
someDao.acquireLock(id); // ConstraintViolationException might be thrown on commit
Data data = otherDao.find(id);
return convert(data);
}
My problem is that locking can only fail on transaction commit which occurs outside of my service method so I have no opportunity to translate the ConstraintViolationException to my custom LockException.
Option 1
One option that's been suggested is to make the service delegate to another method that's #Transactional. E.g.
public Result checkout(String id) throws LockException {
try {
return someInternalService.checkout(id);
}
catch (ConstraintViolationException ex) {
throw new LockException();
}
}
...
public class SomeInternalService {
#Transactional
public Result checkout(String id) {
someDao.acquireLock(id);
Data data = otherDao.find(id);
return convert(data);
}
}
My issues with this are:
There is no reasonable name for the internal service that isn't already in use by the external service since they are essentially doing the same thing. This seems like an indicator of bad design.
If I want to reuse someInternalService.checkout in another place, the contract for that is wrong because whatever uses it can get a ConstraintViolationException.
Option 2
I thought of maybe using AOP to put advice around the service that translates the exception. This seems wrong to me though because checkout needs to declare that it throws LockException for clients to use it, but the actual service will never throw this and it will instead be thrown by the advice. There's nothing to prevent someone in the future from removing throws LockException from the interface because it appear to be incorrect.
Also, this way is harder to test. I can't write a JUnit test that verifies an exception is thrown without creating a spring context and using AOP during the tests.
Option 3
Use manual transaction management in checkout? I don't really like this because everything else in the application is using the declarative style.
Does anyone know the correct way to handle this situation?
There's no one correct way.
A couple more options for you:
Make the DAO transactional - that's not great, but can work
Create a wrapping service - called Facade - whose job it is to do exception handling/wrapping around the transactional services you've mentioned - this is a clear separation of concerns and can share method names with the real lower-level service
When people say "test only one thing". Does that mean that test one feature at a time or one scenario at a time?
method() {
//setup data
def data = new Data()
//send external webservice call
def success = service.webserviceCall(data)
//persist
if (success) {
data.save()
}
}
Based on the example, do we test by feature of the method:
testA() //test if service.webserviceCall is called properly, so assert if called once with the right parameter
testB() //test if service.webserviceCall succeeds, assert that it should save the data
testC() //test if service.webserviceCall fails, assert that it should not save the data
By scenario:
testA() //test if service.webserviceCall succeeds, so assert if service is called once with the right parameter, and assert that the data should be saved
testB() //test if service.webserviceCall fails, so again assert if service is called once with the right parameter, then assert that it should not save the data
I'm not sure if this is a subjective topic, but I'm trying to do the by feature approach. I got the idea from Roy Osherove's blogs, but I'm not sure if I understood it correct.
It was mentioned there that it would be easier to isolate the errors, but I'm not sure if its overkill. Complex methods will tend to have lots of tests.
(Please excuse my wording on the by feature/scenario, I'm not sure how to word them)
You are right in that this is a subjective topic.
Think about how you want this method to behave, not just on how it's currently implemented. Otherwise your tests will just mirror the production code and will break everytime the implementation changes.
Based on the limited context provided, I'd write the following (separate) tests:
Is the webservice command called with the expected data?
If the command returns successfully, is the data saved? Don't overspecify the arguments provided to your webservice call here, as the previous test covers this.
If it's important that the data is not saved when the command returns a failure, I'd write a third test for this. If it's not important, I wouldn't even bother.
You might have heard the adage "one assert per test". This is good advice in general because a test stops executing as soon as a single assert fails. All asserts further down are not executed. By splitting up the asserts in multiple tests you will receive more feedback when something goes wrong. When tests go red, you know exactly all the asserts that fail and don't have to run through the -fix assertion failure, run tests, fix next assertion failure, repeat- cycle.
So in the terminology you propose, my approach would also be to write a test per feature of the method.
Sidenote: you construct your data object in the method itself and call the save method of that object. How do you sense that the data is saved in your tests?
I understand it like this:
"unit test one thing" == "unit test one behavior"
(After all, it is the behavior that the client wants!)
I would suggest that you approach your testing "one feature at a time". I agree with you where you quoted that with this approach it is "easier to isolate the errors". Roy Osherove really does know what he is talking about especially when it comes to TDD.
In my experience I like to focus on the behaviors that I am trying to test (and I am not particularly referring to BDD here). Essentially I would test each behavior that I am expecting from this code. You said that you are mocking out the dependencies (webservice, and data storage) so I would still class this as a unit test with the following expected behaviors:
a call to this method will result in a particular call to a web service
a successful web service call will result in the data being saved
an unsuccessful web service call will result in the data not being saved
Having tests for these three behaviors will help you isolate any issues with the code immediately.
Your tests should also have no dependency on the actual code written to achieve the behavior. For example, if my implementation called some decorator internal to my class which in turn called the webservice correctly then that should be no concern of my test. My test should only be concerned with the external dependencies and public interface of the class itself.
If I exposed internal methods of my class (or implementation details, such as the decorator mentioned above) for the purposes of testing its particular implementation then I have created brittle tests that will fail when the implementation changes.
In summary, I would recommend that your tests should lock down the behavior of a class and isolate failures to identify the 'unit of behavior' that is failing.
A unit test in general is a test that is done without a call to database or file system or even to that effect doesnot call a webservice either. The idea of a unit test is that if you did not have any internet connection you should be able to unit test. So having said that , if a method calls a webservice or calls a database, then you basically are expected to mock the responses from an external system. You should be testing that unit of work only. As mentioned above by prgmtc on how you should be asserting one assert per method is the way to go.
Second, if you are calling a real webservice or database etc, then consider calling those test as integrated or integration test depending upon what you are trying to test.
In my opinion to get the most out of TDD you want to be doing test first development. Have a look at uncle Bobs 3 Rules of TDD.
If you follow these rules strictly, you end up writing tests that generally only have a single assert statements. In reality you will often find you end up with a number of assert statements that act as a single logical assert as it often helps with the understanding of the unit test itself.
Here is an example
[Test]
public void ValidateBankAccount_GivenInvalidAccountType_ShouldReturnValidationFailure()
{
//---------------Set up test pack-------------------
const string validBankAccount = "99999999999";
const string validBranchCode = "222222";
const string invalidAccountType = "99";
const string invalidAccoutTypeResult = "3";
var bankAccountValidation = Substitute.For<IBankAccountValidation>();
bankAccountValidation.ValidateBankAccount(validBankAccount, validBranchCode, invalidAccountType)
.Returns(invalidAccoutTypeResult);
var service = new BankAccountCheckingService(bankAccountValidation);
//---------------Assert Precondition----------------
//---------------Execute Test ----------------------
var result = service.ValidateBankAccount(validBankAccount, validBranchCode, invalidAccountType);
//---------------Test Result -----------------------
Assert.IsFalse(result.IsValid);
Assert.AreEqual("Invalid account type", result.Message);
}
And the ValidationResult class that is returned from the service
public interface IValidationResult
{
bool IsValid { get; }
string Message { get; }
}
public class ValidationResult : IValidationResult
{
public static IValidationResult Success()
{
return new ValidationResult(true,"");
}
public static IValidationResult Failure(string message)
{
return new ValidationResult(false, message);
}
public ValidationResult(bool isValid, string message)
{
Message = message;
IsValid = isValid;
}
public bool IsValid { get; private set; }
public string Message { get; private set; }
}
Note I would have unit tests the ValidationResult class itself, but in the test above I feel it gives more clarity to include both Asserts.
I'm writing unit tests (using NUnit & Moq) for my MVC 2 controllers, and am following examples in the Pro ASP.net MVC 2 Framework book by Steven Sanderson (great book, btw). However, I've run into problems, which I think are just due to my lack of understanding of NUnit.
Here's an excerpt, with the irrelevant parts removed:
[Test]
public void Cannot_Save_Invalid_Event()
{
...
repository.Setup(x => x.SaveEvent(evt)).Callback(Assert.Fail);
...
repository.Verify(x => x.SaveEvent(evt));
}
This test is passing for me, although from what I understand, those two statements should directly conflict with each other. The second one wasn't there originally, but I put it in to verify that it was passing for the right reasons.
From what I understand, my repository is set up to fail if "repository.SaveEvent(evt)" is called. However, later in the test, I try to verify that "repository.SaveEvent(evt)" was called. Since it passes, doesn't this mean that it was both called, and not called? Perhaps those statements don't act as I suspect they do.
Can someone explain how these two statements are not opposites, and how they can both exist and the test still pass?
Maybe your tests doesn-t fail beacuse it has a catch-everything block that also hides the assert/verify-exception that is necessary for the test to fail.
Note: the following unittest will allways pass
[Test]
public void HidingAssertionFailure()
{
try {
Assert.AreEqual(0,1); // this should fail
} catch (Exception ex) {
// this will hide the assertion failure
}
}
The reason for this behavior was that it was running "SaveEvent()", however, since the mocked repository didn't define that action, it was throwing an exception in my controller, which my controller was catching.
So, it seems that the callback will only execute if control returns successfully.
I could have sworn that I've used NUnit's Assert.Throws to determine whether or not a particular exception gets thrown from a method, but my memory has failed me before. I read this post here on SO, but it didn't answer my question, as I know the correct syntax, and I don't want to do anything with the exception that gets returned (I don't want to look at the Exception's members, though this could be useful down the road).
I wrote unit tests to prove my lack of understanding in the use of Dictionary, and couldn't get it handle the KeyNotFoundException that gets thrown. Instead of NUnit catching it and passing the test, I get an unhandled KeyNotFoundException error when I run. I verified that I don't have the VS IDE set up to break on thrown .NET exceptions.
I've tried this two ways:
Assert.Throws( typeof(KeyNotFoundException), () => value = prefs["doesn't exist"]);
and
Assert.Throws<KeyNotFoundException>( () => value = prefs["doesn't exist"]);
but both result in an unhandled exception. What am I missing here?
UPDATE seems like others can't reproduce this. Here's a screenshot:
This is an old thread, but try turning off Enable Just My Code in Visual Studio under Tools->Options. With that on, the debugger is trying to be helpful and stops at the last possible point within your code before the exception gets swallowed.
Or, at least that's my understanding of it.
If you turn off Enable Just My Code, the Assert.Throws should work correctly.
The debugger is stating that your exception is not being handled by user code, which is technically true. To demonstrate, I'll use the sample test sgreeve provided
[Test]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
Assert.Throws(typeof(KeyNotFoundException), () => value = test["h"]);
}
When you execute it, you will receive a warning in VisualStudio that the exception is unhandled in user code. If you look at the callstack, you will see something like
[External Code]
CodeTests.DLL!CodeTests.MiscTests.demonstrateThatExceptionThrown.AnonymousMethod()
[External Code]
CodeTests.DLL!CodeTests.MiscTests.demonstrateThatExceptionThrown()
[External Code]
Because you have specified a delegate, the exception is happening within the "AnonymousMethod" that was created. This is being called by the .Net framework. The debugger is stopping because your delegate isn't handling the exception before it gets passed back to the framework. It doesn't care that further up the stack it might be handled in your code (perhaps since there is no way to guarantee that the external code will handle the exception correctly.)
To have VisualStudio see this as a handled exception, use the ExpectedException attribute and remove the delegate, like so:
[Test]
[ExpectedException(typeof(KeyNotFoundException))]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
value = test["h"];
}
Not a direct answer, but I personally prefer to tag my tests with
[ExpectedException(typeof(KeyNotFoundException))]
public Test ShouldDoTheStuff() {
...
}
Does this work for you? I don't actually see anything wrong with your code per se.
EVEN MORE RECENTLY UPDATED ANSWER!
After our conversation in the comments added to this answer, I suspect that the nunitit test runner is the problem here. I don't believe there's anything wrong with your test as I have no problem executing it either using NUnit GUI or the excellent Resharper test runner.
UPDATED ANSWER
After seeing your screen shot, I tried stepping through my test with the debugger and saw exactly the same prompt about the unhandled exception. If I carry on stepping past that error, the test passes when I reach the end of the assertion.
When I run the test in none-debug mode using either the NUnit GUI or the Resharper 4.5 test runner, the test passes as expected every time.
Sorry to ask the obvious question, but what are you executing your test with? i.e. which test runner?
The exact code I've executed is:
using System;
using System.Collections.Generic;
using NUnit.Framework;
namespace ClassLibrary1
{
[TestFixture]
public class DictionaryTest
{
[Test]
public void demonstrateThatExceptionThrown()
{
string value;
Dictionary<string, string> test = new Dictionary<string, string>();
Assert.Throws(typeof(KeyNotFoundException), () => value = test["h"]);
}
}
}