I am using StoryQ to perform some basic integration testing and we are using NHibernate as our ORM.
When I started, I didn't know that NHibernate implemented the Repository pattern and so I created my own IRepository in order to run my integration tests.
However, considering that NHibernate already implements the Repository pattern, I assume that it is doing so against some kind of interface. So, I would like to work against NHibernate's interface for the Repository if my assumptions are correct.
I have tried to search for it but I come across information that to do that I need to work against the ISession interface. As I do not really know NHibernate that well, can someone explain why I would need to implement my fake repository against the ISession interface? What is the IRepository equivalent in NHibernate? Is there some tutorial which goes into greater depth into the matter?
NHibernate doesn't implement the Repository pattern. It replaces it.
SQLite in-memory databases are nice if you've got a simple database implementation, but I've found that things can become cumbersome quickly, almost to a point where it becomes as painful, if not more, to use SQLite as it is to stub/mock ISession/ICriteria/etc.
One perfect example of this: In one of my recent projects, in which I was using PostgreSQL as my production database and SQLite as my test database, I had a need to extend NHibernate to add support for an aggregate function that was recently added to PostgreSQL. Figuring out how to add this was a story in itself, but I worked it out. I then had to find a functional equivalent in SQLite. I needed an aggregate function that worked in the same exact way as its Postgres counterpart. There was none. I asked around and was told that there were ways to extend NHibernate to "fake" this function in SQLite. I also had the option of extending SQLite to add this functionality.
All I wanted to do was write two, maybe three, tests around the scenario that I was trying to implement. I ended up spending way too much time trying to ensure functional equivalency between the two systems. It wasn't worth all this effort for one function. And what would happen if down the road, I needed to add another function?
I think SQLite is useful. It's a great lightweight database system and I love that you can conveniently use it as a in-memory database for simple scenarios. However, I'm not sure it's worth using beyond that. I think from now on, I'll be using the same database across all environments, even if it means having slower integration tests for all data persistence logic.
I'm not sure where in core NHibernate there is an IRepository interface (AFAIK there's none) so you might be referring some other NHibernate side projects.
It is not the best approach to go about mocking ISession either. The best thing in my opinion, is to use a real in-memory database that is fully supported by NHibernate. You may need to check how to configure NHibernate to run on sqlite in-memory database, which basically is just configuring NHibernate in your tests.
The good thing about this approach, is that tests run with a very good speed, as if there's no database involved, and you don't need to abstract away all your ORM functionality (and loose features too) just to run / drive your tests.
The approach I've taken is to implement an IRepository and GenericRepository, to wrap the ISession Save and Delete methods. Also, GenericRepository also implements IQueryable using NHibernate's LINQ provider. My implementation borrows heavily from Javier Lozano's MVC Trubine project. His implementation is here: https://github.com/jglozano/mvcturbine/blob/master/src/Blades/NHibernate/MvcTurbine.NHibernate/GenericRepository.cs
This works for about 80% of the queries I need. For the rest, I create a Query Object to wrap the query. That way, I can use ICriteria, IQuery, IQueryOver or LINQ, whichever the needs of the query dictate. Fabio Maulo explains it pretty well here: http://fabiomaulo.blogspot.com/2010/07/enhanced-query-object.html
Both approaches allow easily mocking away the dependency.
Related
As i dig deeper in to the DbContext, DbSet and associated interfaces, I am wondering why you would need to implement a separate "Generic" Repository around these implementations?
It looks like DbContext and IDbSet do everything you need and include the "Unit Of Work" inside DbContext.
Am I missing something here or does it seem people enjoy adding another layer of dependency for no reason.
You are actually right. DbContext is an implementation of the unit of work pattern and IDbSet is an implementation of the repository pattern.
Repositories are currently very popular and overused. Everybody use them just because there are dozens of articles about creating repository for entity framework but nobody actually describes challenges related to this decision.
Main reasons for using repository are usually:
Hide EF from upper layer
Make code better testable
The first reason is some kind of architectonic purity and great idea that if you make your upper layers independent on EF you can later on switch to other persistence framework. How many times did you see such thing in the real world? This reason makes working with EF much harder because your repository must expose a lot of additional features wrapping what EF allows by default.
In the same time wrapping EF code can keep your code better organized and following Separation of concern rule. For me this can be the only real advantage of repository and unit of work but you have to understand that following this rule with EF will maybe make your code better maintainable and better readable but in the initial effort to create your application will be much higher and for smaller applications this can be unnecessary complexity.
The second reason is partially correct. The big disadvantage of EF is rigid architecture which can be hardly mocked so if you want to unit test upper layer you must wrap EF somehow to allow mocking its implementation. But this has many other consequences which I described here.
I follow Ayende's blog. If you ever used NHibernate you probably know his articles. This guy recently wrote several articles against using repository with NHibernate but NHibernate is much better mockable.
I am struggling with the same issues, and mockability for unit testing of the EF layers is important. But I ran across this great article which explains how to set up the EF 4.1 DbContext to be mockable by making sure your derived DbContext implemented a generic interface and exposes IDbSet rather than DbSet's. Since I am using a Database First approach, because our database already exists, I simply modified the T4 templates used to generate my derived DbContext to generate it to return IDbSet interfaces, as well as derive from my generic interface. That way the entire thing can be easily mocked, and you don't need to implement your own Unit Of Work or repository pattern. Just write your service code to consume your generic interface, and when you go to unit test it, just mock the generic interface with specific test data and you are good to go.
http://refactorthis.wordpress.com/2011/05/31/mock-faking-dbcontext-in-entity-framework-4-1-with-a-generic-repository/
One reason for creating the repository is so you can hide the implementation of DBSet and DbContext if you decide to move from EntityFramework to something else or vice versa.
For example, I was using NHibernate and I wrapped all of the calls to that framework inside my repository classes. They return IEnumerable for their gets to be "generic" and my repositories have the standard CRUD operations (update, delete, etc). I have long since moved to Entity Framework. Upon doing so, I did not need to change anything in my ViewModel classes or beyond because they pointed to my repository--I only needed to change the inside of my repository. This made life much easier when migrating.
(I used NHibernate because we are connecting to the ISeries, and at the time, there were no cost affective implementations using EF with the ISeries. The only one available was to pay $12,000 to IBM for their DB2Connect)
What's the advantage of using mock objects in comparison to a static test database that has known data and using transactions to make sure nothing changes when testing against the database.
You can do both. Use mock objects to test you BLL logic and then use a test database to test your DAL logic. That way if something breaks you can easily see where the problem lies by which test fails.
Firstly, using a mock will be much quicker than connecting to an external database. However the main reason is that mocks behave the same each time the test is run, which you can't guarentee with an external service like a database, which means unit tests don't fail randomly. You can also easily simulate any types of failure you want to handle with a mock object.
It's also a good idea to run integration tests against a real database however to test things like configuration and performance.
If you can ensure that the static test db doesn't change during testing then I think static test db is better than mock objects. But, then it depends on what you want to test and the complexity of the code being tested. I would go with mock objects as they are simpler to maintain compared with a db.
Imagine you're about to write a class which doesn't exist. Maybe it's a controller. It's not something which talks straight to the database.
You have a good idea of how it ought to behave, and you know what it should be responsible for, and what it should delegate to some other service (using Single Responsibility Principle). So you write interfaces to represent the roles of the helper classes it's going to use.
Then you write an example of how you might use your class that you're about to create. If you like, you can call the example a unit test. You mock out the interactions with the helper classes. When you run the example, it fails, because you haven't written the code yet. You can now write the code to make it pass, using the interfaces of the helper classes - which you also haven't written yet.
Then you do the same with the helper classes, mocking out their helpers.
Eventually you'll reach a class which talks to the database. Or maybe it talks to a web service. Or perhaps the data is static, or in memory. (It doesn't matter to the original class, because your class is decoupled from this).
At this point you'll want an example to describe the behaviour of this class, too, and if it's a database connector, you'll need a database for the example.
Writing code this way produces code that's easy to use and understand, with nothing extra that isn't needed by something further up the stack. This is usually more robust than code that's easier to write. That's why we sometimes use mocks - because writing tests first using mocks helps produce good, maintainable, decoupled design.
Usually you would use both of these approaches.
For unit tests you would use mocked objects. This way you can test your system in a much more fine-grained way, because you can mock every object - not only the ones that wrap the database connection. In general it is good practise to unit test every class in separation of all the dependencies. The benefits are - its is possible to test all the error handling on all levels and make sure you test all the code paths, when you get a test failure you can immediately find the cause, etc.
For integration and end-to-end tests you test big parts of the system or the whole thing. Then you would connect to the database (because this connection is part of the test). Obviously you have to ensure that the database is in a known state, etc., etc.
You will end up having much more unit tests than integration tests, so its actually quite important to make them very quick - yet another advantage of using mock objects.
The setup for testing with a db can be arduous. If you find yourself spending more time in setting up the db, just to test some functional aspect of your business logic, then you might want to use a mock/fake.
I've read a few threads on SO about usefulness of unit-testing various applications. The opinions can range from "test everything all the time" to "unit tests are useless", and everything in between ("test where it makes sense"). I tend to lean towards the middle one.
That leads me to my question. I am trying to decide if it would be beneficial or practical to have some basic unit-tests testing 3rd party ORM as suggested in this SO post:
link text
some baseline tests may be useful as insurance against future breaking changes, depending on how you are using the tool. For example, instead of mocking up the entire n-tier chain (I'm not a fan of mocking when it is not necessary), just use the ORM tool to create, read, update, and delete a typical object/record, and verify the operation using direct SQL statements on the (test) database. That way if the 3rd-party vendor later updates something that breaks the basic functionality you'll know about it, and new developers to your project can easily see how to use the ORM tool from the unit test examples.
My main reservations following this advise is that it would require way too much setup, would be a headache to maintain, and over all it would not be practical in our environment. Here's the summary of some points to consider:
The ORM we're using requires static datasource object(s) to be created and registered with its Data Access Layer and associated with authenticated user. This would require a lot of test setup, and probably would be problematic on the build server where no user is logged on.
ORM vendor has a pretty good track record of releasing new updates and not breaking basic functionality. Furthermore whenever it's time to update ORM to the latest version, I would imagine that application wouldn't go straight to production, but would be thoroughly regression tested anyway.
Maintaining test db for Unit testing is kind of problematic in this environment. Test db gets wiped out after each major release and replaced with db backup from staging with sensative data obfuscated. I would imagine in order to have a test db for ORM unit testing, we would need to run some scripts/code that would set the database in a "test" state. Again too much setup and maintenance.
And finally ORM documentation/help for new developers. I can see how something like that could be useful. But ORM vendor provides pretty good documentation/help with demo apps. So writing unit tests on top of that doesn't seem to be worth all the efforts.
So, is it worth to go through all these troubles just to make sure that ORM does what it supposed to do (which is CRUD)? Shouldn't it be a responsibility of the vendor anyway?
You said it yourself. Test where it makes sense. If it will make you "feel" better to test the 3rd party ORM, then do it. But, ultimately, you're putting your trust in their tool. What are you going to do if the ORM suddenly stops working? Have you written enough code against it that you can't easily rip it out? You'd wait for them to fix it, probably.
Basically, you have to treat 3rd party tools as the proverbial black boxes and let them do what you bought them to do. That was the reason you paid the money you did, right? To keep from having to write that yourself.
In this particular case, I wouldn't bother. I think you are correct in assuming a bad ROI here.
And yes, I consider it the responsibility of the vendor. I expect (and assume) their stuff works. That's how I treat my vendors.
It is the responsibility of the vendor to make sure the ORM does what it's supposed to do, but it's your responsibility to ensure that your application does what it's supposed to do, and if it fails for whatever reason, your clients will be unhappy, even if it's "just" because the ORM failed.
Your tests could ensure that the ORM works the way you expect it to given the way you're calling it. It's possible that the ORM will change in a way that isn't "broken" but that doesn't play nicely with your application.
That being said, if you're confident in the ORM, and feel that setting up and maintaining any kind of automated tests of the ORM is not worth the effort, it's probably not, especially if you've got other levels of testing that are likely to reveal the problems if they arise.
I personally think that real unit tests should only test the application itself, and everything that needs to be separately deployed and configured should be mocked up.
What you are saying is to write some integration/functional tests, that test the whole system end-to-end. These will never be lightweight, but probably are still useful in some cases (e.g. if your system doesn't change too much and is critical for your company at the same time). I have seen such tests automated as well, using virtual servers (either VMWare or microsoft equivalent), and an example database which was restored from file before every test run. You can also just set the ORM once, and accept that the tests will fail mainly because the configuration will break. Obviously you can test more, but be aware that the cost is higher.
Testing that 3rd party ORM library does its job is not unit testing at all. However, that's not the point of your question.
As was said numerous times in such a books like "Working Effectively with Legacy Code" by Michael Feathers or "Domain-Driven Design" by Eric Evans or "Clean Code" by Robert Martin your 3rd party ORM library is a technical detail which should be abstracted away from your codebase, precisely because you have no control over 3rd party libraries by definition. If they change, you accommodate.
So your solution is to make a wrapper around this ORM library, publishing ideally domain-related interface to rest of your application but generic interface is probably will do, too. This wrapper you need to test using fullstack automated tests, which inevitably should setup your application along with the database and all configuration and preparation required for it. This tests are not unit-level and expected to be really slow.
You can read about the different levels of tests and how they should be set up in the chapter 6 of the book "Continuous Integration" by Paul M. Duvall.
When writing true unit-level tests for your application level, you mock the wrapper above the ORM library, which you are able to do because you control the code of the wrapper.
This is a standard practice. The obvious benefit of it is that when you decide to update the ORM library or (which is highly possible) when your client/boss decide to switch to another ORM or database which this ORM is not compatible with, you will have the instant feedback about regression errors from the tests of your wrapper and all you'll need to do is to accommodate to changes inside your wrapper.
"Too much maintenance burden" is a fallacy created by lack of automation, by the way.
Recently there has been quite some hype around all the different mocking frameworks in the .NET world. I still haven't quite grasped what is so great about them. It doesn't seem to be to hard to write the mocking objects I need myself. Especially with the help of Visual Studio I quickly can write a class that implements the interface I want to mock (it auto-generates almost everything for me) and then write an implementation for the method(s) I need for my test. Done! Why going through the hassle of understanding a mocking framework for the sole purpose of saving a few lines of code. Or is a mocking framework not only about saving lines of code?
Once I finally got the hang of mock objects, I realized that they're essential for unit testing for the same reason that double blind testing or control groups are essential for scientific trials: they isolate what you're actually testing.
If you're testing a class which has quite a bit of interaction via other interfaces, you not only save the lines of code on having to mock each and every interface, but you also gain the ability to do things like "throw an exception if an unexpected method is called" or "exception if these methods are called out of order". You can get remarkably sophisticated with mock frameworks, and though I'll quickly admit there's a large learning curve, when you get up to speed they'll help make your unit tests more thorough without being bloated.
You actually identified one of the key points of a mock framework in your question. The fact that you code the mocks yourself is not something the developer should be concerned with. The mocking frameworks give you implementations of interfaces programatically, plus they are functional (based on your setup of the mock).
What do you do if you are testing an ICustomerDAO, for example, and you want to test some method 14 times each with different outcomes? Implement 14 different classes manually? I doubt that anyone would want to do that.
Mocks give you the power to define what will happen with parts of your classes when you are not concerned with whether or not they will actually work, like throwing exceptions whenever you want them to, returning zero results and making sure you handle that correctly, etc...
They are a great unit testing tool.
Previous questions that may help:
What is a mock and when should you use it?
Mockist vs classical TDD
I find that using a mocking framework allows me to generate tests a lot faster and with better verification that what I expect to happen in the test actually is happengin. I have in the past implemented stubs or fakes myself. I found that I needed to generate stubs specific to the test that I wanted and this took a lot of time. I can create the same test much faster using a mocking framework. The good ones support the generation of fakes, stubs or mocks with straightforward syntax.
It takes a while to get the hang of it, I avoided it for a while but now wouldn't try to work without a mocking framework for the reasons #Chamelaeon states.
Roy Osherove had a poll about Mock Frameworks and down in the comment section, there is a discussion (albeit brief) about whether one needs a Mock Framework or not.
I personally have been manually doing exactly as you stated and it has worked well enough, but this has mainly been out of habit rather than a closely-held opinion on mock frameworks in general.
Well I certainly don't think that you NEED a mocking framework. It's a framework like any other, and it's ultimately designed to save you some time and effort. You can also do things like roll your own common data structures like stacks and queues, but isn't it generally easier to just use the ones built into the class libraries that ship with the compiler/IDE of your language of choice?
I'm sure there are other compelling reasons for using mocking frameworks, though I'd leave it to the TDD and unit testing gurus to answer.
For the same reason you wouldn't try to write unit tests without NUnit. A mocking framework will assist you in verifying state and behavior over hundreds of unit tests. It's worth the 2 weeks or so of pain to get up to speed and really helps you focus on what needs to be tested.
One thing that troubles me about a mocking framework is that "what a function should o/p given an i/p" via
when(mock.someMethod("some arg")).thenReturn("something");
statement is spread across many unit test classes.
Let me elaborate with an example. Lets say there was a DAO Interface function getEmp(int EmpID) which was returning an Employee Object when passed an Employee ID as a parameter. Assume that this function was being mocked by 10 different unit test classes. Now if in the future, this function were changed to return a newer version of the Employee Object, one would have to go to each of the 10 different classes to update this change.
The disadvantages are as follows...
a) I don't know how to figure out all the classes which mock this function so that I can go update this change.
b) My existing test cases which consumes the mock DAO object continue to be blissfully unaware of the changes that have happened to the DAO Interface because the mock has not changed and hence continue to be green.
Ideally, if I were to have coded a single mock class myself and consumed it everywhere, I would have just one place to update for the newer version of the Employee object. Also, once I update at this one place, all my existing test cases which consume the mock would break and I would then know exactly what places I need to go and do an update for the new Employee Object.
Any thoughts on my views..
One of the good things about a mocking framework is that it allows setting expectations on the objects being mocked. With the expectations I can then set up all sorts of conditions to exercise the code thats being tested.
An isolation framework or mocking framework allows you to test the code you want, without its dependencies. It makes for short running tests, allows you to debug quickly, and easily build a safety net of tests around the code. Different frameworks have different features, and as said before - it's a tool, and you should select the right tool for the job.
I've use rhino mocks for a mocking framework. I and 5 other developers used it on a large enterprise application that was an 8 month project. We used tdd on the project. Was it worth it? I guess. Was there such a massive huge selling point to using mocks that I have to use it on every project? In my opinion, no. It is not something that is necessary, it is just a tool that you can use if you want to try it out. Some projects you can roll out your own mock classes as some here say they prefer - it is easier. Other projects are larger and may require a mocking framework. The key word (in my opinion) is MAY require... how much code coverage do you require? To me, that is another consideration to using mocks. The project I did with tdd/rhino mocks we were required to have 80% code coverage so the mocks helped us attain that. If our code coverage requirements were less, for example 40%, we probably would have not used a mocking framework and just wrote our own mock classes as others mention they do.
I am curious about how many of you folks incorporate mocking of objects (frameworks like JMock, NMock, RhinoMocks hand in hand with unit testing frameworks) into your daily development process. What are your experiences?
You see, I develop on top of a GIS (geographic information systems) platform, in which most of work pertains to some way of handling data. As its data object model is quite complex (many many classes and interfaces, all COM-based), it is also quite difficult and cumbersome to mock. In this case, mocking incurs a great deal of overhead when writing test suites. I wonder if there are people in similar situation, or just, how does mocking (in whatever situation you are in) work for you.
On a recent project that I worked on we used mock objects extensively in our unit testing approach. The project was 100% Java and moderately sized (about 100,000 lines of non-commented code). It was a Swing-based desktop application - and the only effective way that we found to test the user interface logic was through an MVC variant design that allowed us to use mock objects to substitute for the actual Swing user interface classes for the automated testing. We also used mocking extensively in the testing of our data access layer (Hibernate/DAOs).
In the user inteface use, the Mocks were easy and straightforward to build. And the design of the application (Fowler Passive View) easily incorporated mocks. This was not the case for the mocks used in testing the data access layer. But I can say that it was clearly worth the effort. In fact, most of the 'effort' really focused on coming up with a reusable solution that minimized the work that a developer had to do to create each individual mock. I'd recommend taking the time to dig into and discover an approach for your situation that allows you to easily mock up your GIS data layer. That - or just manually mock up each class. Either way the ability to run the automated unit tests that rely on the mocks is worthwhile...
In my situation mocks work really nice. But I'm using Python, which is so dynamic that it makes many things involving testing much, much easier.
In situation like yours, when application is mainly data-driven (as far as I see), mocks may not be as useful. Just passing data in and watching it come out should be enough for testing. I would just make sure that application is modularized enough, so this approach can be applied to reasonably small components.
Mocking can be useful in some kind of project. But, sometimes mocking is very time consuming and the ROI of it is low.
Trying to test Sharepoint it seems that mocking is the only way, and only typemock will let you mock sealed classes.
Mocking is used very extensively in my case. Mocks are usually for the classes that has external dependencies, e.g. network, database, filesystem. Any of these can introduce flakiness in the tests if mocks are not used.
If the mocks that you find costly to write because there are a lot of fake data to populate, you could set some pre-populated data objects as constants and use them or slightly modified copies in your test. If such data objects has external dependencies, then maybe refactor it in a way you can separate the two concerns.
There is an initiative started by Dave Bouman to try and build a community library of Mocks for use in ArcObjects related unit testing. His blog and this svn repository have great information related to unit testing GIS systems
http://blog.davebouwman.net/CategoryView,category,Unit%2BTesting.aspx
http://svn2.assembla.com/svn/arcdeveloper/TestingUtilities/trunk/