Application Service Layer: Unit Tests, Integration Tests, or Both? - unit-testing

I've got a bunch of methods in my application service layer that are doing things like this:
public void Execute(PlaceOrderOnHoldCommand command)
{
var order = _repository.Load(command.OrderId);
order.PlaceOnHold();
_repository.Save(order);
}
And at present, I have a bunch of unit tests like this:
[Test]
public void PlaceOrderOnHold_LoadsOrderFromRepository()
{
var repository = new Mock<IOrderRepository>();
const int orderId = 1;
var order = new Mock<IOrder>();
repository.Setup(r => r.Load(orderId)).Returns(order.Object);
var command = new PlaceOrderOnHoldCommand(orderId);
var service = new OrderService(repository.Object);
service.Execute(command);
repository.Verify(r => r.Load(It.Is<int>(x => x == orderId)), Times.Exactly(1));
}
[Test]
public void PlaceOrderOnHold_CallsPlaceOnHold()
{
/* blah blah */
}
[Test]
public void PlaceOrderOnHold_SavesOrderToRepository()
{
/* blah blah */
}
It seems to be debatable whether these unit tests add value that's worth the effort. I'm quite sure that the application service layer should be integration tested, though.
Should the application service layer be tested to this level of granularity, or are integration tests sufficient?

I'd write a unit test despite there also being an integration test. However, I'd likely make the test much simpler by eliminating the mocking framework, writing my own simple mock, and then combining all those tests to check that the the order in the mock repository was on hold.
[Test]
public void PlaceOrderOnHold_LoadsOrderFromRepository()
{
const int orderId = 1;
var repository = new MyMockRepository();
repository.save(new MyMockOrder(orderId));
var command = new PlaceOrderOnHoldCommand(orderId);
var service = new OrderService(repository);
service.Execute(command);
Assert.IsTrue(repository.getOrder(orderId).isOnHold());
}
There's really no need to check to be sure that load and/or save is called. Instead I'd just make sure that the only way that MyMockRepository will return the updated order is if load and save are called.
This kind of simplification is one of the reasons that I usually don't use mocking frameworks. It seems to me that you have much better control over your tests, and a much easier time writing them, if you write your own mocks.

Exactly: it's debatable! It's really good that you are weighing the expense/effort of writing and maintaining your test against the value it will bring you - and that's exactly the consideration you should make for every test you write. Often I see tests written for the sake of testing and thereby only adding ballast to the code base.
As a guideline I usually take that I want a full integration test of every important successful scenario/use case. Other tests I'll write are for parts of the code that are likely to break with future changes, or have broken in the past. And that is definitely not all code. That's where your judgement and insight in the system and requirements comes into play.
Assuming that you have an (integration) test for service.Execute(placeOrderOnHoldCommand), I'm not really sure if it adds value to test if the service loads an order from the repository exactly once. But it could be! For instance when your service previously had a nasty bug that would hit the repository ten times for a single order, causing performance issues (just making it up). In that case, I'd rename the test to PlaceOrderOnHold_LoadsOrderFromRepositoryExactlyOnce().
So for each and every test you have to decide for yourself ... hope that helps.
Notes:
The tests you show can be perfectly valid and look well written.
Your test sequence methods seems to be inspired on the way the Execute(...) method is currently implemented. When you structure your test this way, it could be that you are tying yourself to a specific implementation. This way, tests can actually make it harder to change - make sure you're only testing the important external behavior of your class.

I usually write a single integration test of the primary scenario. By primary scenario i mean the successful path of all the code being tested. Then I write unit tests of all the other scenarios like checking all the cases in a switch, testing exception and so forth.
I think it is important to have both and yes it is possible to test it all with integration tests only, but that makes your tests long running and harder to debug. In average I think I have 10 unit tests per integration test.
I don't bother to test methods with one-liners unless something bussines logic-like happens in that line.
Update: Just to make it clear, cause I'm doing test-driven development I always write the unit tests first and typically do the integration test at the end.

Related

Functional tests are superset of unit tests, are they?

I have been reading about unit tests and functional tests for a while now.
If I write exhaustive functional tests, won't they in-turn cover the units underneath as well, which in-turn make the unit tests redundant?
We follow agile and we write the functional tests using WebDriver as soon as we are done with the 'slice' of the functionality, which is typically 2-4 weeks of sprint time.
Perhaps, but not necessarily.
Functional tests can be thought of as "Black Box" tests whereby you are looking at a specific function (regardless if that's a single method, module, system, etc) and checking that for a given input, you get a given output.
However, if the functional test fails all you can say is that the system is faulty; it doesn't necessarily give you any indication of what part of the system is to blame. Of course you will go off and diagnose the problem but you don't know up front what the problem is.
e.g
//assuming you have a UserService that amongst other things passes through to a UserRepository
var repo = new UserRepository();
var sut = new UserService(repo);
var user = sut.GetUserByID(1);
Assert.IsNotNull(user); //Suppose this fails.
In the above case, you don't know if it's because the UserService's GetUserByID() function is faulty - perhaps it did not call repo.GetUserByID and just returned null, perhaps it did get a User from repo but accidentally then returned an uninitialized temp variable, etc - or perhaps it's because the dependency (repo) itself is faulty. In any case, you'll have to debug into the issue.
Unit tests, on the other hand, are more like "White Box" tests whereby you have effectively taken the cover off the system and are testing how the system behaves in order to do what you want it to do.
e.g. (code below may not compile and is just for demo purposes)
//setup a fake repo and fake the GetUserByID method to return a known instance:
var repo = new Mock<UserRepository>();
var user = new User{ID=1;}
repo.Setup(r=>r.GetUserByID(1).Returns(user);
var sut = new UserService(repo.Object);
var actual= sut.GetUserByID(1);
//now make sure we got back what we expected. If we didn't then UserService has a problem.
Assert.IsNotNull(user);
Assert.AreEqual(user,actual);
In this case you are explicitly verifying that sut.GetUserByID() behaves in the expected way - that it invokes the repo.GetUserByID() and returns resultant object, without mangling it or changing it.
If this unit test passes, but the functional test fails, you can say with confidence that the problem does not lie with the UserService, but with the UserRepository class, without even looking at the code. That may or may not save you time, it really depends on how complex your functional units are.

Unit Testing and Integration Testing examples

I have created following four tests in a Test class that tests a findCompany() method of a CompanyService.
#Test
public void findCompany_CompanyIdIsZero() {
exception.expect(IllegalArgumentException.class);
companyService.findCompany(0);
}
#Test
public void findCompany_CompanyIdIsNegative() {
exception.expect(IllegalArgumentException.class);
companyService.findCompany(-100);
}
#Test
public void findCompany_CompanyIdDoesntExistInDatabase() {
Company storedCompany = companyService.findCompany(100000);
assertNull(storedCompany1);
}
#Test
public void findCompany_CompanyIdExistsInDatabase() {
Company company = new Company("FAL", "Falahaar");
companyService.addCompany(company);
Company storedCompany1 = companyService.findCompany(company.getId());
assertNotNull(storedCompany1);
}
My understanding says that the first three of these are unit tests. They test the behavior of the findCompany() method, checking how the method will respond on different inputs.
The fourth test, though placed in the same class, actually seems to be an integration test to me. It requires a Company to be added to the database first, so that it can be found later on. This introduces external dependencies - addCompany() and database.
Am I going right? If yes, then how should I unit test finding an existing object? Just mock the service to "find" one? I think that kills the intent of the test.
I appreciate any guidance here.
I look at it this way: the "unit" you are testing here is the CompanyService. In this sense all of your tests look like unit tests to me. Underneath your service, though, there may be another service (you mention a database) that this test is also exercising? This could start to blur the lines with integration testing a bit, but you have to ask yourself if it matters. You could stub out any such underlying service, and you may want to if:
The underlying service is slow to set up or use, making your unit tests too slow.
You want to be sure the behaviour of this test is unaffected by the underlying service - i.e. this test should only fail if there is a bug in CompanyService.
In my experience, provided the underlying service is fast enough I don't worry too much about my unit test relying on it. I don't mind a bit of integration leaking into my unit tests, as it has benefits (more integration coverage) and rarely causes a problem. If it does cause problems you can always come back to it and add stubbing to improve the isolation.
[1,2,3,4] could be unit-based (mocked | not mocked) and integration-based tests. It depends what you want to test.
Why use mocking? As Jason Sankey said ...test only service tier not underlaying tier.
Why use mocking? Your bussiness logic can have most various forms. So you can write several test for one service method, eg. create person (no address - exception, no bank account - exception, person does not have filled not-null attributes - exception).
Can you imagine that each test requested database in order to test all possibility exception states (no adress, no bank account etc.)? There is too much work to fill database in order to test all exception states. Why not to use mocked objects which eg. act like 'crippled' objects which do not contains expected values. Each test construct own 'crippled' mock object.
Mocking various states === your test will be simply as possible because each test method will be test only one state. These test will be clear and easy to understand and maintance. This is one of goals which I want to reach if I write a test.

How to choose TDD starting point in a real world project?

I've read tons of articles, seen tons of screencasts about TDD, but I'm still struggling with using it in real world project. My main issue is I don't know where to start, what test should be the first one.
Suppose I have to write client library calling external system's methods (e.g. notification).
I want this client to work as follows
NotificationClient client = new NotificationClient("abcd1234"); // client ID
Response code = client.notifyOnEvent(Event.LIMIT_REACHED, 100); // some params of call
There is some translation and message format preparation behind the scenes, so I'd like to hide it from my client apps.
I don't know where and how to start.
Should I make up some rough classes set for this library?
Should I start with testing NotificationClient as below
public void testClientSendInvalidEventCommand() {
NotificationClient client = new NotificationClient(...);
Response code = client.notifyOnEvent(Event.WRONG_EVENT);
assertEquals(1223, code.codeValue());
}
If so, with such test I'm forced to write complete working implementation at once, with no baby steps as TDD states. I can mock out sosmething in Client but then I have to know this thing to be mocked upfront, so I need some upfront desing to be made.
Maybe I should start from the bottom, test this message formatting component first and then use it in right client test?
What way is the right one to go?
Should we always start from top (how to deal with this huge step required)?
Can we start with any class realizing tiny part of desired feature (as Formatter in this example)?
If I'd know where to hit with my tests it'd be a lot easier for me to proceed.
I'd start with this line:
NotificationClient client = new NotificationClient("abcd1234"); // client ID
Sounds like we need a NotificationClient, which needs a client ID. That's an easy thing to test for. My first test might look something like:
public void testNewClientAbcd1234HasClientId() {
NotificationClient client = new NotificationClient("abcd1234");
assertEquals("abcd1234", client.clientId());
}
Of course, it won't compile at first - not until I'd written a NotificationClient class with a constructor that takes a string parameter and a clientId() method that returns a string - but that's part of the TDD cycle.
public class NotificationClient {
public NotificationClient(string clientId) {
}
public string clientId() {
return "";
}
}
At this point, I can run my test and watch it fail (because I've hard-coded clientId()'s return to be an empty string). Once I've got my failing unit test, I write just enough production code (in NotificationClient) to get the test to pass:
public string clientId() {
return "abcd1234";
}
Now all my tests pass, so I can consider what to do next. The obvious (well, obvious to me) next step is to make sure that I can create clients whose ID isn't "abcd1234":
public void testNewClientBcde2345HasClientId() {
NotificationClient client = new NotificationClient("bcde2345");
assertEquals("bcde2345", client.clientId());
}
I run my test suite and observe that testNewClientBcde2345HasClientId() fails while testNewClientAbcd1234HasClientId() passes, and now I've got a good reason to add a member variable to NotificationClient:
public class NotificationClient {
private string _clientId;
public NotificationClient(string clientId) {
_clientId = clientId;
}
public string clientId() {
return _clientId;
}
}
Assuming no typographical errors have snuck in, that'll get all my tests to pass, and I can move on to whatever the next step is. (In your example, it would probably be testing that notifyOnEvent(Event.WRONG_EVENT) returns a Response whose codeValue() equals 1223.)
Does that help any?
Don't confuse acceptance tests that hook into each end of your application, and form an executable specifications with unit tests.
If you are doing 'pure' TDD you write an acceptance test which drives the unit tests that drive the implementation. testClientSendInvalidEventCommand is your acceptance test, but depending on how complicated things are you will delegate the implementation to multiple classes you can unit test separately.
How complicated things get before you have to split them up to test and understand them properly is why it is called Test Driven Design.
You can choose to let tests drive your design from the bottom up or from the top down. Both work well for different developers in different situations. Either approach will force to make some of those "upfront" design decisions but that's a good thing. Making those decisions in order to write your tests is test-driven design!
In your case you have an idea what the high level external interface to the system you are developing should be so let's start there. Write a test for how you think users of your notification client should interact with it and let it fail. This test is the basis for your acceptance or integration tests and they are going to continue failing until the features they describe are finished. That's ok.
Now step down one level. What are the steps which need to occur to provide that high level interface? Can we write an integration or unit test for those steps? Do they have dependencies you had not considered which might cause you to change the notification center interface you have started to define? Keep drilling down depth-first defining behavior with failing tests until you find that you have actually reached a unit test. Now implement enough to pass that unit test and continue. Get unit tests passing until you have built enough to pass an integration test and so on. You'll eventually have completed a depth-first construction of a tree of tests and should have a well tested feature whose design was driven by your tests.
One goal of TDD is that the testing informs the design. So the fact that you need to think about how to implement your NotificationClient is a good thing; it forces you to think of (hopefully) simple abstractions up front.
Also, TDD sort of assumes constant refactoring. Your first solution probably won't be the last; so as you refine your code the tests are there to tell you what breaks, from compile errors to actual runtime issues.
So I would just jump right in and start with the test you suggested. As you create mocks, you will need to create tests for the actual implementations of what you are mocking. You will find things make sense and need to be refactored, so you will need to modify your tests as you go. That's the way it's supposed to work...

Unit Testing.... a data provider?

Given problem:
I like unit tests.
I develop connectivity software to external systems that pretty much and often use a C++ library
The return of this systems is nonndeterministic. Data is received while running, but making sure it is all correctly interpreted is hard.
How can I test this properly?
I can run a unit test that does a connect. Sadly, it will then process a life data stream. I can say I run the test for 30 or 60 seconds before disconnecting, but getting code ccoverage is impossible - I simply dont even comeclose to get all code paths EVERY ONCE PER DAY (error code paths are rarely run).
I also can not really assert every result. Depending on the time of the day we talk of 20.000 data callbacks per second - all of which are not relly determined good enough to validate each of them for consistency.
Mocking? Well, that would leave me testing an empty shell of myself because the code handling the events basically is the to be tested case, and in many cases we talk here of a COMPLEX c level structure - hard to have mocking frameworks that integrate from Csharp to C++
Anyone any idea? I am short on giving up using unit tests for this part of the application.
Unit testing is good, but it shouldn't be your only weapon against bugs. Look into the difference between unit tests and integration tests: it sounds to me like the latter is your best choice.
Also, automated tests (unit tests and integration tests) are only useful if your system's behavior isn't going to change. If you're breaking backward compatibility with every release, the automated tests of that functionality won't help you.
You may also want to see a previous discussion on how much unit testing is too much.
Does your external data source implement an interface -- or can you using a combination of an interface and a wrapper around the data source that implements the interface decouple your class under test from the data source. If either of these are true, then you can mock out the data source in your unit tests and provide the data from the mock instance.
public interface IDataSource
{
public List<DataObject> All();
...
}
public class DataWrapper : IDataSource
{
public DataWrapper( RealDataSource source )
{
this.Source = source;
}
public RealDataSource Source { get; set; }
public List<DataObject> All()
{
return this.Source.All();
}
}
Now in your class under test depend on the interface and inject an instance, then in your unit tests, provide a mock instance that implements the interface.
public void DataSourceAllTest()
{
var dataSource = MockRepository.GenerateMock<IDataSource>();
dataSource.Expect( s => s.All() ).Return( ... mock data ... );
var target = new ClassUnderTest( dataSource );
var actual = target.Foo();
// assert something about actual
dataSource.VerifyAllExpectations();
}

MEF and unit testing with NUnit

A few weeks ago I jumped on the MEF (ComponentModel) bandwagon, and am now using it for a lot of my plugins and also shared libraries. Overall, it's been great aside from the frequent mistakes on my part, which result in frustrating debugging sessions.
Anyhow, my app has been running great, but my MEF-related code changes have caused my automated builds to fail. Most of my unit tests were failing simply because the modules I was testing were dependent upon other modules that needed to be loaded by MEF. I worked around these situations by bypassing MEF and directly instantiating those objects.
In other words, via MEF I would have something like
[Import]
public ICandyInterface ci { get; set; }
and
[Export(typeof(ICandyInterface))]
public class MyCandy : ICandyInterface
{
[ImportingConstructor]
public MyCandy( [Import("name_param")] string name) {}
...
}
But in my unit tests, I would just use
CandyInterface MyCandy = new CandyInterface( "Godiva");
In addition, the CandyInterface requires a connection to a database, which I have worked around by just adding a test database to my unit test folder, and I have NUnit use that for all of the tests.
Ok, so here are my questions regarding this situation:
Is this a Bad Way to do things?
Would you recommend composing parts in [SetUp]
I haven't yet learned how to use mocks in unit testing -- is this a good example of a case where I might want to mock the underlying database connection (somehow) to just return dummy data and not really require a database?
If you've encountered something like this before, can you offer your experience and the way you solved your problem? (or should this go into the community wiki?)
It sounds like you are on the right track. A unit test should test a unit, and that's what you do when you directly create instances. If you let MEF compose instances for you, they would tend towards integration tests. Not that there's anything wrong with integration tests, but unit tests tend to be more maintainable because you test each unit in isolation.
You don't need a container to wire up instances in unit tests.
I generally recommend against composing Fixtures in SetUp, as it leads to the General Fixture anti-pattern.
It is best practice to replace dependencies with Test Doubles. Dynamic mocks is one of the more versatile ways of doing this, so definitely something you should learn.
I agree that creating the DOCs manually is much better than using MEF composition container to satisfy imports, but regarding the note 'compositing fixtures in setup leads to the general fixture anti pattern' - I want to mention that that's not always the case.
If you’re using the static container and satisfy imports via CompositionInitializer.SatisfyImports you will have to face the general fixture anti pattern as CompositionInitializer.Initialize cannot be called more than once. However, you can always create CompositionContainer, add catalogs, and call SatisyImportOnce on the container itself. In that case you can use a new CompositionContainer in every test and get away with facing the shared/general fixture anti pattern
I blogged on how to do unit tests (not nunit but works just the same) with MEF.
The trick was to use a MockExportProvider and i created a test base for all my tests to inherit from.
This is my main AutoWire function that works for integration and unit tests:
protected void AutoWire(MockExportProvider mocksProvider, params Assembly[] assemblies){
CompositionContainer container = null;
var assCatalogs = new List<AssemblyCatalog>();
foreach(var a in assemblies)
{
assCatalogs.Add(new AssemblyCatalog(a));
}
if (mocksProvider != null)
{
var providers = new List<ExportProvider>();
providers.Add(mocksProvider); //need to use the mocks provider before the assembly ones
foreach (var ac in assCatalogs)
{
var assemblyProvider = new CatalogExportProvider(ac);
providers.Add(assemblyProvider);
}
container = new CompositionContainer(providers.ToArray());
foreach (var p in providers) //must set the source provider for CatalogExportProvider back to the container (kinda stupid but apparently no way around this)
{
if (p is CatalogExportProvider)
{
((CatalogExportProvider)p).SourceProvider = container;
}
}
}
else
{
container = new CompositionContainer(new AggregateCatalog(assCatalogs));
}
container.ComposeParts(this);
}
More info on my post: https://yoavniran.wordpress.com/2012/10/18/unit-testing-wcf-and-mef/