I'm writing a client built on top of a mongo.Client from the "mongo" package. Because of the nature of mongo client which requires a live database connection, I've decided to try the mocking approach, that is, to store an interface with a set of methods (which also return the interfaces) I would use on the mongo client.
The issue is that the return type of some mongo functions doesn't implement my interfaces for a reason which I do not understand.
For example, mongo.Collection doesn't implement my ICollection interface, however, the mongo.SingleResult does implement the ISingleResult interface:
type ICollection interface {
FindOne(context.Context, interface{}, ...*options.FindOneOptions) ISingleResult
}
type ISingleResult interface {
Decode(v interface{}) error
}
The issue is gone if I change the return type from ISingleResult to *mongo.SingleResult, but this is not what I want, because I could not control the *mongo.SingleResult logic and track its calls as I could do with its mock.
Please help me to fix that logic or provide another testing approach which is more common if my is wrong.
Related
I am having a hard time figuring out an idiomatic way of writing testable code in golang. I understand the importance of interfaces and their use in testing, but I haven't figured out how to mock/test external struct dependencies.
As an example, I have written the following which simulates a wrapper for creating a pull request on GitHub.
type GitHubService interface {
}
type gitHubService struct {
CreatePullRequest(...) (PullRequest,error)
}
func (s gitHubService) CreatePullRequest(...) (PullRequest,error) {
tp := github.BasicAuthTransport{
Username: strings.TrimSpace(/*.....*/),
Password: strings.TrimSpace(/*.....*/),
}
client := github.NewClient(tp.Client())
pr,err := client.Repositories.CreatePullRequest(...)
...
}
func TestPullRequest(t *testing.T) {
service := gitHubService{}
pr,err := service.CreatePullRequest(...)
...
}
If I was writing a unit test for GitHubService.CreatePullRequest(...) I would want to mock the call to client.Repositories.CreatePullRequest(...) and probably even github.NewClient(...) to return mock implementations that I can control.
With tools such as gomock it seems that you are out of luck with structs and package functions.
What is the idiomatic way to handle this? I am very familiary with Inversion of Control and the different patterns such as Dependency Injection and Service Locator, but I have heard countless times that this is not idiomatic.
One important design feature of Go is decoupling (Watch this great talk from Bill Kennedy about that topic). Inside your method there are some dependencies, which could be decoupled. This coupled method makes it not really testable.
Thing you should refactor:
tp := github.BasicAuthTransport: you should not initialize the authorization inside of your method. It should move into your gitHubService as a parameter. Inside your method call you can access ist via s.tp. You could also make it an input parameter of the method.
github.NewClient() and client.Repositories.CreatePullRequest(...) just read about the golang best practices from Peter Bourgon Make dependencies explicit!. The alternative is to create an interface, which contains all the called functions. This interface should be an input to your method.
After your code is decoupled you can mock everything very easy. If you use interfaces as an input you can just create a mock struct, which implements the interface. If you make the dependencies explicit you can overwrite them. In the last case the code for storing the values of the calls is not so clean, but it also works. The idiomatic go way is to use interfaces.
I had a similar problem and looks like, the only option you have is to have another layer in between which would call your "unmockable" clients.
For e.g. for mocking the govmi clients (vmware clients sdk for golang), I had to have a "myCustomClient" having interfaces and structs to make calls to govmi.Client.AnyMethod..
I could then generate mocks for "myCustomClient".
mockgen -source myCustomClient.go -package myPackage -destination myCustomClientMock.go
You can install it by: got get github.com/golang/mock
I have seen lots of posts (and debates!) about which way round UnitOfWork and Repository. One of the repository patterns I favor is the typed generic repository pattern, but I fear this had lead to some issues with clean code and testability. Take the following repository interface and generic class:
public interface IDataEntityRepository<T> : IDisposable where T : IDataEntity
{
// CRUD
int Create(T createObject);
// etc.
}
public class DataEntityRepository<T> : IDataEntityRepository<T> where T : class, IDataEntity
{
private IDbContext Context { get; set; }
public DataEntityRepository (IDbContext context)
{
Context = context;
}
private IDbSet<T> DbSet { get { return Context.Set<T>(); } }
public int Create(T CreateObject)
{
DbSet.Add(createObject);
}
// etc.
}
// where
public interface IDbContext
{
IDbSet<T> Set<T>() where T : class;
DbEntityEntry<T> Entry<T>(T readObject) where T : class;
int SaveChanges();
void Dispose();
}
So basically I am using the Context property in each pattern to gain access to the underlying context.
My problem is now this: when I create my unit of work, it will effectively be a wrapper of the context I need the repository to know about. So, if I have a Unit Of Work that declares the following:
public UserUnitOfWork(
IDataEntityRepository<User> userRepository,
IDataEntityRepository<Role> roleRepository)
{
_userRepository = userRepository;
_roleRepository = roleRepository;
}
private readonly IDataEntityRepository<User> _userRepository;
public IDataEntityRepository<User> UserRepository
{
get { return _userRepository; }
}
private readonly IDataEntityRepository<Role> _roleRepository;
public IDataEntityRepository<Role> RoleRepository
{
get { return _roleRepository; }
}
I have a problem with the fact that the two repositories I am passing in both need to be instantiated with the very Unit Of Work into which they are being passed. Obviously I could instantiate the repositories inside the constructor and pass in the "this" but that tightly couples my unit of work to a particular concrete instance of the repositories and makes unit testing that much harder.
I would be interested to know if anyone else has headed down this path and hit the same wall. Both these patterns are new to me so I could well be doing something fundamentally wrong. Any ideas would be much appreciated!
UPDATE (response to #MikeSW)
Hi Mike, many thanks for your input. I am working with EF Code First but I wanted to abstract certain elements so I could switch to a different data source or ORM if required and because I am (trying!) to push myself down a TDD route and using Mocking and IOC. I think I have realised the hard way that certain elements cannot be unit tested in a pure sense but can have integration tests! I'd like to raise your point about Repositories working with business objects or viewmodels etc. Perhaps I have misunderstood but if I have what I see as my core business objects (POCOs), and I then want to use an ORM such as EF code first to wrap around those entities in order to create, and then interact with, the database (and, it's possible, I may re-use these entities within a ViewModel), I would expect a Repository to handle these entities directly in the context of some set of CRUD operations. The entities know nothing about the persistence layer at all, neither would any ViewModel. My unit of work simply instantiates and holds the required repositories allowing a transaction commit to be performed (across multiple repositories but the same context/ session). What I have done in my solution is to remove the injection of an IDataEntityRepository ... etc. from the UnitOfWork constructor as this is a concrete class that must know about one and only one type of IDataEntityRepository it should be creating (in this case DataEntityRepository, which really should be bettered names as EFDataEntityRepository). I cannot unit test this per se because the whole unit logic would be to establish the repositories with a context (itself) to some database. It simply needs an integration test. Hope that makes sense?!
To avoid dependency on each repository in your Unit of Work, you could use a provider based on this contract:
public interface IRepositoryProvider
{
DbContext DbContext { get; set; }
IRepository<T> GetRepositoryForEntityType<T>() where T : class;
T GetRepository<T>(Func<DbContext, object> factory = null) where T : class;
void SetRepository<T>(T repository);
}
then you could inject it into your UoW that would look like this:
public class UserUnitOfWork: IUserUnitOfWork
{
public UserUnitOfWork(IRepositoryProvider repositoryProvider)
{
RepositoryProvider = repositoryProvider;
}
protected IDataEntityRepository<T> GetRepo<T>() where T : class
{
return RepositoryProvider.GetRepositoryForEntityType<T>();
}
public IDataEntityRepository<User> Users { get { return GetRepo<User>(); } }
public IDataEntityRepository<Role> Roles { get { return GetRepo<Role>(); } }
...
Apologies for the tardiness of my response - I have been trying out various approaches to this in the mean time. I have marked up the answers above because I agree with the comments made.
This is one of those questions where there is more than one answer and it's very much dependent upon the overall approach. Whilst I agree that EF effectively provides a ready-made unit of work pattern, my decision to create my own unit of work and repository layers was to be able to control access to the database entities.
Where I struggled was in the need to be able to inject a repository into a unit of work. What I realised though was that in the case of EF, my unit of work was effectively a thin wrapper around multiple repositories with a Commit (SaveChanges) method. It was not responsible for executing specific actions such as FindCustomer etc.
So I decided that a unit of work could be tightly coupled to its specific type of DataRepository pattern. To ensure I had a testable pattern, I introduced a service layer that provided the facade for executing particular actions such as CreateCustomer, FindCustomers etc. These services that accepted an IUnitOfWork constructor parameter which provided access to the repositories (as interfaces) as well as the Commit method.
I was then able to create fakes of both unit of work and/ or repositories for testing purposes. This just left me with the decision of what could be unit tested with fakes and what needed to be integration tested with the concrete instances.
And this also gives me the opportunity to control what actions are performed on the database and how they are performed.
I'm sure there are many ways to skin this particular cat but the goals of provided a clean interface that is testable have been just about met with this approach.
My thanks to g1ga and Mike for their input.
When using Entity Framework (EF) (which I assume you're using) you already have a generic repository IDbSet. It's useless to ad another layer on top just to call EF methods.
Also, a repository works with application objects (usually business objects, but they can be view models or objects state). If you're just using db entities, you kinda defeat the Repository pattern purpose ( to isolate the business bojects from the database). THe original pattern deals only with busines objects, but it is a useful pattern outside the business layer too.
The point is that EF entities are Persistence objects and have (or should have) no relation to your business objects. You want to use the repository pattern to 'translate' the busines objects to persistence objects and viceversa.
Sometimes it might happen that an application object (like a viewmodel) to be the same with a persistence entity (and in that case you can use directly EF objects) but that's a coincidence.
About Unit of Work (UoW), let's say that's tricky. Personally, I prefer to use the DDD (domain driven design) approach and consider that any business object (BO) sent to the repoistory is a UoW, so it will be wrapped in a transaction.
If I need to update multiple BOs, I'll use a message driven architecture to send commands to the relevant BOs. Of course, that's more complicated and requires to be at ease with the concept of eventual consistency but I'm not depending on a specific RDBMS.
If you know that you'll be using a specific RDBMS and that will never be changed, you could start a transaction and pass the associated connection to each repository, with a commit at the end (that will be the UoW). If you're in a web setting, it's even easier, start transaction when the request begins, commit when requests ends (you can use an ActionFilter for ASp.Net Mvc).
However this solution is tied up to one RDBMS, so it won't apply to a NoSql or any storage which doesn't support transactions. For those cases, the message driven way is the best.
I have a business rule visual studio class library (.NET 2.0) project that takes a dependency on Dynamics Crm Web Services - a classic SOAP web reference as opposed to a WCF endpoint. I want to unit test those business rules without having a real crm instance behind it. Adding a web reference doesn't produce an interface that I can fake. It does generate c# in my project that I think I can fake if I can create the interface. I don't think I need to actually navigate HTTP and get into all of the protocol stuff.
I saw Joh Skeet's blog post. Alas I didn't want to write any code and I'm hoping a tool has been written since then that might help. I tried some of his steps but concluded that he is smarter than me and I couldn't make that work.
I am aware of SoapUI, however, I was hoping for pure unit tests that would work in a CI build environment.
Is there a way to do this.
The standard way to mock something which doesn't come with an interface, is to build your own wrapper around it.
the code you want to mock, say the webservice stuff:
class AutoGeneratedStuff
{
public string GeneratedMethodYouUse()
{...}
public string GeneratedMethodYouDontNeed()
{...}
}
you then make an interface which covers only the bits of the code you need:
public interface IWebServiceClient
{
string MethodYouUse();
}
and a concrete wrapper class which implements it, which has a dependency to the generated stuff
class WebServiceClient : IWebServiceClient
{
private AutoGeneratedStuff _stuff;
public WebService(AutogeneratedStuff stuff)
{
_stuff = stuff;
}
public string MethodYouUse()
{
return _stuff.MethodYouUse();
}
}
then, in your code when you would have called the generated class, call your interface instead. In your unit tests, you can mock the interface, either using a mocking framework, or by implementing the interface with another concrete class that has no dependencies to the generated stuff
I've researched some information about techniques I could use to unit test a DbContext. I would like to add some in-memory data to the context so that my tests could run against it. I'm using Database-First approach.
The two articles I've found most usefull were this and this.
That approach relies on creating an IContext interface that both MyContext and FakeContext will implement, allowing to Mock the context.
However, I'm trying to avoid using repositories to abstract EF, as pointed by some people, since EF 4.1 already implements repository and unit of work patterns through DbSet and DbContext, and I really would like to preserve all the features implemented by the EF Team without having to maintain them myself with a generic repository, as I already did in other project (and it was kind of painful).
Working with an IContext will lead me to the same path (or won't it?).
I thought about creating a FakeContext that inherits from main MyContext and thus take advantage of the DbContext underneath it to run my tests without hitting the database.
I couldn't find similar implementations, so I'm hoping someone can help me on this.
Am I doing something wrong, or could this lead me to some problems that I'm not anticipating?
Ask yourself a single question: What are you going to test?
You mentioned FakeContext and Mocking the context - why to use both? Those are just different ways to do the same - provide test only implementation of the context.
There is one more bigger problem - faking or mocking context or set has only one result: You are not testing your real code any more.
Simple example:
public interface IContext : IDisposable
{
IDbSet<MyEntity> MyEntities { get; }
}
public class MyEntity
{
public int Id { get; set; }
public string Path { get; set; }
}
public class MyService
{
private bool MyVerySpecialNetMethod(e)
{
return File.Exists(e.Path);
}
public IEnumerable<MyEntity> GetMyEntities()
{
using (IContext context = CreateContext())
{
return context.MyEntities
.Where(e => MyVerySpecialNetMethod(e))
.Select(e)
.ToList();
}
}
}
Now imagine that you have this in your SUT (system under test - in case of unit test it is an unit = usually a method). In the test code you provide FakeContext and FakeSet and it will work - you will have a green test. Now in the production code you will provide a another derived DbContext and DbSet and you will get exception at runtime.
Why? Because by using FakeContext you have also changed LINQ provider and instead of LINQ to Entities you are running LINQ to Objects so calling local .NET methods which cannot be converted to SQL works as well as many other LINQ features which are not available in LINQ to Entities! There are other issues you can find with data modification as well - referential integrity, cascade deletes, etc. That is the reason why I believe that code dealing with context / LINQ to Entities should be covered with integration tests and executed against the real database.
I am developing an open-source library to solve this problem.
http://effort.codeplex.com
A little teaser:
You don't have to add any boilerplate code, just simply call the appropriate API of the library, for example:
var context = Effort.ObjectContextFactory.CreateTransient<MyContext>();
At first this might seem to be magic, but the created ObjectContext object will communicate with an in-memory database and will not talk to the original real database at all. The term "transient" refers to the lifecycle of this database, it only lives during the presence of the created ObjectContext object. Concurrently created ObjectContext objects communicate with dedicated database instances, the data is not shared accross them. This enables to write automated tests easily.
The library provides various features to customize the creation: share data across instances, set initial data of the database, create fake database on different data layers... check out the project site for more info.
As of EF 4.3, you can unit test your code by injecting a fake DefaultConnectionFactory before creating the context.
Entity Framework 4.1 is close to being able to be mocked up in tests but requires a little extra effort. The T4 template provides you with a DbContext derived class that contains DbSet properties. The two things that I think you need to mock are the DbSet objects that these properties return and properites and methods you're using on the DbContext derived class. Both can be achieved by modifying the T4 template.
Brent McKendrick has shown the types of modifications that need to be made in this post, but not the T4 template modifications that can achieve this. Roughly, these are:
Convert the DbSet properties on the DbContext derived class into IDbSet properties.
Add a section that generates an interface for the DbContext derived class containing the IDbSet properties and any other methods (such as SaveChanges) that you'll need to mock.
Implement the new interface in the DbContext derived class.
Short version of my questions:
Can anyone point me toward some good, detailed sources from which I
can learn how to implement testing in my MVC 3 application, using
NUnit, Ninject 2, and Moq?
Can anyone here help clarify for me how Controller-Repository
decoupling, mocking, and dependency injection work together?
Longer version of my questions:
What I'm trying to do ...
I am currently beginning to create an MVC 3 application, which will use Entity Framework 4, with a database first approach. I want to do this right, so I am trying to design the classes, layers, etc., to be highly testable. But, I have little to no experience with unit testing or integration testing, other than an academic understanding of them.
After lots of research, I've settle on using
NUnit as my testing framework
Ninject 2 as my dependency injection framework
Moq as my mocking framework.
I know the topic of which framework is best, etc., could enter into this, but at this point I really don't know enough about any of it to form a solid opinion. So, I just decided to go with these free solutions which seem to be well liked and well maintained.
What I've learned so far ...
I've spent some time working through some of this stuff, reading resources such as:
Implementing the Repository and Unit of Work Patterns in an
ASP.NET MVC Application
Building Testable ASP.NET MVC Applications
NerdDinner Step 12: Unit Testing
Using Repository and Unit of Work patterns with Entity Framework
4.0
From these resources, I've managed to workout the need for a Repository pattern, complete with repository interfaces, in order to decouple my controllers and my data access logic. I have written some of that into my application already, but I admit I am not clear as to the mechanics of the whole thing, and whether I am doing this decoupling in support of mocking, or dependency injection, or both. As such, I certainly wouldn't mind hearing from you guys about this too. Any clarity I can gain on this stuff will help me at this point.
Where things got muddy for me ...
I thought I was grasping this stuff pretty well until I started trying to wrap my head around Ninject, as described in Building Testable ASP.NET MVC Applications, cited above. Specifically, I got completely lost around the point in which the author begins describing the implementation of a Service layer, about half way into the document.
Anyway, I am now looking for more resources to study, in order to try to get various perspectives around this stuff until it begins to make sense to me.
Summarizing all of this, boiling it down to specific questions, I am wondering the following:
Can anyone point me toward some good, detailed sources from which I
can learn how to implement testing in my MVC 3 application, using
NUnit, Ninject 2, and Moq?
Can anyone here help clarify for me how Controller-Repository
decoupling, mocking, and dependency injection work together?
EDIT:
I just discovered the Ninject official wiki on Github, so I'm going to start working through that to see if it starts clarifying things for me. But, I'm still very interested in the SO community thoughts on all of this :)
If you are using the Ninject.MVC3 nuget package, then some of the article you linked that was causing confusion will not be required. That package has everything you need to start injecting your controllers which is probably the biggest pain point.
Upon installing that package, it will create a NinjectMVC3.cs file in the App_Start folder, inside that class is a RegisterServices method. This is where you should create the bindings between your interfaces and your implementations
private static void RegisterServices(IKernel kernel)
{
kernel.Bind<IRepository>().To<MyRepositoryImpl>();
kernel.Bind<IWebData>().To<MyWebDAtaImpl>();
}
Now in your controller you can use constructor injection.
public class HomeController : Controller {
private readonly IRepository _Repo;
private readonly IWebData _WebData;
public HomeController(IRepository repo, IWebData webData) {
_Repo = repo;
_WebData = webData;
}
}
If you are after very high test coverage, then basically anytime one logical piece of code (say controller) needs to talk to another (say database) you should create an interface and implementation, add the definition binding to RegisterService and add a new constructor argument.
This applies not only to Controller, but any class, so in the example above if your repository implementation needed an instance of WebData for something, you would add the readonly field and the constructor to your repository implementation.
Then when it comes to testing, what you want to do is provide mocked version of all required interfaces, so that the only thing you are testing is the code in the method you are writing the test for. So in my example, say that IRepository has a
bool TryCreateUser(string username);
Which is called by a controller method
public ActionResult CreateUser(string username) {
if (_Repo.TryCreateUser(username))
return RedirectToAction("CreatedUser");
else
return RedirectToAction("Error");
}
What you are really trying to test here is that if statement and the return types, you do not want to have to create a real repository that will return true or false based on special values you give it. This is where you want to mock.
public void TestCreateUserSucceeds() {
var repo = new Mock<IRepository>();
repo.Setup(d=> d.TryCreateUser(It.IsAny<string>())).Returns(true);
var controller = new HomeController(repo);
var result = controller.CreateUser("test");
Assert.IsNotNull(result);
Assert.IsOfType<RedirectToActionResult>(result)
Assert.AreEqual("CreatedUser", ((RedirectToActionResult)result).RouteData["Action"]);
}
^ That won't compile for you as I know xUnit better, and do not remember the property names on RedirectToActionResult from the top of my head.
So to sum up, if you want one piece of code to talk to another, whack an interface in between. This then allows you to mock the second piece of code so that when you test the first you can control the output and be sure you are testing only the code in question.
I think it was this point that really made the penny drop for me with all this, you do this not necessarily becase the code demands it, but because the testing demands it.
One last piece of advice specific to MVC, any time you need to access the basic web objects, HttpContext, HttpRequest etc, wrap all these behind an interface as well (like the IWebData in my example) because while you can mock these using the *Base classes, it becomes painful very quickly as they have a lot of internal dependencies you also need to mock.
Also with Moq, set the MockBehaviour to Strict when creating mocks and it will tell you if anything is being called that you have not provided a mock for.
Here is the application that I'm creating. It is open source and available on github, and utilizes all of the required stuff - MVC3, NUnit, Moq, Ninject - https://github.com/alexanderbeletsky/trackyt.net/tree/master/src
Contoller-Repository decoupling is simple. All data operations are moved toward the Repository. Repository is an implementation of some IRepository type. The controller never creates repositories inside itself (with the new operator) but rather receives them either by constructor argument or property.
.
public class HomeController {
public HomeController (IUserRepository users) {
}
}
This technique is called "Inversion of Control." To support inversion of control you have to provide some "Dependency Injection" framework. Ninject is a good one. Inside Ninject you associate some particular interface with an implementation class:
Bind<IUserRepository>().To<UserRepository>();
You also substitute the default controller factory with your custom one. Inside the custom one you delegate the call to the Ninject kernel:
public class TrackyControllerFactory : DefaultControllerFactory
{
private IKernel _kernel = new StandardKernel(new TrackyServices());
protected override IController GetControllerInstance(
System.Web.Routing.RequestContext requestContext,
Type controllerType)
{
if (controllerType == null)
{
return null;
}
return _kernel.Get(controllerType) as IController;
}
}
When the MVC infrastructure is about to create a new controller, the call is delegated to the custom controller factory GetControllerInstance method, which delegates it to Ninject. Ninject sees that to create that controller the constructor has one argument of type IUserRepository. By using the declared binding, it sees that "I need to create a UserRepository to satisfy the IUserRepository need." It creates the instance and passes it to the constructor.
The constructor is never aware of what exact instance would be passed inside. It all depends on the binding you provide for that.
Code examples:
https://github.com/alexanderbeletsky/trackyt.net/blob/master/src/Web/Infrastructure/TrackyServices.cs https://github.com/alexanderbeletsky/trackyt.net/blob/master/src/Web/Infrastructure/TrackyControllerFactory.cs https://github.com/alexanderbeletsky/trackyt.net/blob/master/src/Web/Controllers/LoginController.cs
Check it out : DDD Melbourne video - New development workflow
The whole ASP.NET MVC 3 development process was very well presented.
The third party tools I like most are:
Using NuGet to install Ninject to enable DI throughout the MVC3
framework
Using NuGet to install nSubstite to create mocks to enable unit
testing