I'm having hard times writing unit tests in Go due to external libraries which don't expose an interface (therefore not mockable) but only pure functions. Even big ones like Google don't, so I'm wondering whether my approach is good enough.
Wouldn't be good practice for libraries to provide interfaces instead of packages with only functions in order to let the user mock them?
The solution I came up with until now is wrap these packages with an interface's implementation but that seem like too much work.
I come with an example. My function could look like this
func AnyFunction() error {
sess := session.Get("blabla")
// logic in here...
}
where session is an imported package that returns a struct. I can't mock the package session.
For this case I'm going to write a SessionInterface with an implementation, which internally calls session.
Ex:
type SessionInterface interface {
Get(s string) Session
}
type mySessionImpl struct {}
func (me *mySessionImpl) Get(s string) Session {
return session.Get(s)
}
For my tests now I can mock the SessionInterface and inject that one into my code
You're doing the right thing here, and this was a conscious decision on the part of the language designers.
The Go philosophy is that your code should "own" those interfaces, not the library. In languages like C# and Java, libraries define their interfaces up front, without really knowing what the consumer actually needs. (Maybe they include too many methods, or too few.) In Go, because the consumer effectively "owns" the interfaces, you're empowered to specify which methods actually need to be present in a minimal interface, and changes in your program requirements mean that you can also change the interface.
Now in this particular case it might seem strange to create an adapter in front of a function for testability, but consider the alternative: if session.Get() were a method of an interface or struct, instead of a function, it would force all library consumers to instantiate a dummy object in order to call the method. Not everyone is going to fake it out---it's easier for them to say that the consumers who want to (like you) are empowered to write adapters, and those that don't can blissfully ignore them.
IMO this is a super common solution and strikes a good balance between maintainability and testability.
The way I think of this is that your code is programming to an interface, and there just happens to be two implementations:
"prod" version ie the library that you're testing
test version, a stub that implements the interface
With this approach, a stub can only verify up to your service and its dependencies boundary along the contract of the interface, it assures very little or nothing about your components collaboration/integration with the library that's being stubbed. In my experiences this approach works awesome and is my personal preference. I have found it to be important though to have 1 or 2 higher level tests making sure that your component can successfully initialize and interact with the "prod" version of the library.
Plug:
I've written about this exact issue https://medium.com/dm03514-tech-blog/you-are-going-to-need-it-using-interfaces-and-dependency-injection-to-future-proof-your-designs-2cf6f58db192
Related
I'm working on a personal project (meaning clean source code, no legacy dependencies) and attempting to follow best practices regarding unit testing, dependency management, etc.
My company's codebase is littered with code like this:
public Response chargeCard(CreditCard card, Money amount) {
if(Config.isInUnitTests()) {
throw new IllegalStateException("Cannot make credit card API calls in unit tests!");
}
return CreditCardPOS.connect().charge(card, amount);
}
The goal here is to actively prevent dangerous code / code with external dependencies from being executed during testing. I like the notion of failing fast if unit tests do something bad, but I don't like this implementation for a few reasons:
It leaves hidden dependencies to the static Config class scattered throughout our codebase.
It changes the control flow between testing and live behavior, meaning we're not necessarily testing the same code.
It adds an external dependency to a configuration file, or some other state-holding utility.
It looks ugly :)
A fair number of the places where we use this in my company's codebase could be avoided by better dependency awareness, which I'm attempting to do, but there are still some places where I'm still struggling to see away around implementing an isInUnitTests() method.
Using the credit card example above, I can avoid the isInUnitTests() check on every charge by properly wrapping this up inside a mock-able CardCharger class, or something similar, but while cleaner I feel like I've only moved the problem up one level - how do I prevent a unit test from constructing a real instance of CardCharger without putting a check in the constructor / factory method that creates it?
Is isInUnitTests() a code smell?
If so, how can I still enforce that unit tests don't reach external dependencies?
If not, what's the best way to implement such a method, and what are good practices for when to use / avoid it?
To clarify, I'm trying to prevent unit tests from accessing unacceptable resources, like the database or network. I'm all for test-friendly patterns like dependency injection, but good patterns are useless if they can be violated by a careless developer (i.e. me) - it seems important to me to fail-fast in cases where unit tests do things they aren't supposed to, but I'm not sure the best way to do that.
Is isInUnitTests() a code smell?
Yes, definitively, you have many ways to avoid coupling your code to unit tests. There is no valid reason to have something like that.
If so, how can I still enforce that unit tests don't reach external dependencies?
Your tests must only validate a unit of code and create mock or stubs for external dependencies.
Your code seems to be Java, which it's plenty of mature mock frameworks. Research a little about existing ones and pick the one that likes you more.
EDIT
how do I prevent a unit test from constructing a real instance of HTTPRequest without putting a check in the constructor / factory method that creates it
You're supposed to use a dependency injector to resolve your instance dependencies so you will never need to use a if to test if you're on a test or not since on your "real" code you inject full functional dependencies and on your test you inject mock or stubs.
Take a more serious example, like a credit card charger (a classic unit testing example) - it would seem like a very poor design if it were even possible for test code to access the real card charger without triggering a deluge of exceptions. I don't think it's enough in a case like that to trust the developer will always do the right thing
Again, you're supposed to inject external dependencies as a credit card charger so on your test you would inject a fake one. If some developer don't know this, I think the first thing your company needs is training for that developer and some pair programming to guide the process.
In any case, kind of I get your point and let me tell you a similar situation I experienced.
There was an application that sent a bunch of emails after some processing. These emails were not to be send on any other environment except live, otherwise it would be a big problem.
Before I started working on this application, it happened several times that developers "forgot" about this rule and emails were sent on test environments, causing a lot of problems. It's not a good strategy to depends on human memory to avoid this kind of problem.
What we did to avoid this to happen again was adding a config setting to indicate if send real emails or not. The problem was broader than executing things or not on unit tests as you can see.
Nothing replace communication though, a developer could set the incorrect value for this setting on his development environment. You're never 100% safe.
So in short:
First line of defense is communication and training.
You second line of defense should be dependency injection.
If you feel that this is not enough, you could add a third line of defense: a configuration setting to avoid executing real logic on test/development environments. Nothing wrong about that. But please don't name it IsInUnitTest as the problem is broader than that (you want also o avoid this logic to be executed on developer machine)
The AngularJS Unit Testing Documentation actually does a really fantastic job of describing the different ways that developers handle unit testing.
Of the four different methods they outline, the one they recommend is one that involves using dependency injection so that your production code flow is the same as your testing flow. The only differences are that the objects you pass into your code would be different. Note that Claudio used the term "Stub" to refer to objects you pass into a method to act as placeholders for those you'd use in production:
public Document getPage(String url, AbstractJsoup jsoup) {
return jsoup.connect(url).get();
}
My Java is a little rusty, so consider that you may need to actually do some funky stuff to make this work, like check the type of the jsoup object or cast it to your test Jsoup object or your production Jsoup object, which might just defeat the entire purpose of using dependency injection. Thus, if you're talking about a dynamic language like JavaScript or some other loosely-typed functional programming language, dependency injection would keep things really clean.
But in Java, unless you control the dependencies and can use a design pattern, like the strategy pattern, to pass in different concrete implementations, doing what you're doing may be easier, especially since you don't control the Jsoup code. However, you might check to see if they have available stubs you can use as placeholders, as some library developers do write stubs.
If you don't own the code, another option could be to use a Factory class to obtain the desired objects, depending on a flag you set when first instantiating it. This seems like a less desirable solution, since you're still sort of setting a global flag in that Factory object that may have an effect on things you might not be trying to test. The advantage of dependency injection for unit testing is that you must explicitly pass in a test stub when testing and explicitly pass in the production object when you wish the method to do what it was written to do. Once the method completes its execution, the test is over, and any production process that invokes it will automatically run it in production mode because it will inject production objects.
I've never seen a system where steps were taken to actively prevent code running under unit tests from accessing external resources. The problem just never came up. You have my deepest sympathy for working somewhere where it has.
Is there a way you could control the classpath used for unit tests, to make the libraries needed to access external resources unavailable? If there is no JSoup and no JDBC driver on the classpath, tests for code which attempts to use them will fail. You won't be able to exclude JDK-level classes like Socket and URLConnection this way, but it might still be useful.
If you were running tests using Gradle, that would be fairly straightforward. If you are using Maven, maybe not. I don't think there's any way to have different classpaths for compilation and testing in Eclipse.
Well, you can achieve the same clean way using Abstract Factory Pattern though (maybe not suitable to call it Abstract Factory Pattern).
Example in C#:
public class CardChargerWrapper{
public CardChargerWrapper(
NullCardCharger nullCharger
, TestUnitConfig config){
// assign local value
this.charger = new CardCharger();
}
CardCharger charger;
NullCardCharger nullCharger;
TestUnitConfig config;
public Response Charge(CreditCard card, Money amount){
if(!config.IsUnitTest()) { return charger.connect().charge(card, amount); }
else { return NullCardCharger.charge(card, amount); }
}
}
EDIT: Changed CardChargerWrapper to use hard-coded instance of CardCharger instead of injecting it.
Note: You can change NullCardCharger to something like MockCardCharger or OfflineCardCharger for logging purpose.
Note again: You can change the CardChargerWrapper's constructor to fit. Example, instead of constructor injecting the NullCardCharger, you can make it property injected. Same with TestUnitConfig.
EDIT: Regarding calling IsUnitTest() a good idea or not:
It really depends on your business perspective and how you are doing testings. As many people said, a code that has not yet been tested is not trusted for their correctness. It cannot be reliabled. On side note, I prefer IsChargeRealCard() will be more suitable than IsUnitTest().
Say that we take out the unit test in our context, at least you will still need to do integration testing in test environment. You probably want to test something like:
I want to test the credit card validation (is it real or not, etc).
I want to test the payment method and see whether the card being charged. As a process, but not as a real credit card payment.
For 2nd point, I think the best thing to do is to create a mock credit card charger to log the transaction. This is, to ensure that the charging is correct. And it will be conducted in both test and dev server.
So, How can the CardChargerWrapper help such situation?
Now with CardChargerWrapper, you can:
Switch the NullCardCharger to any mock card chargers to enhance your unit testing.
All class using CardChargerWrapper can be ensured that they are checking IsUnitTest first before charging real card.
Your developer need to use CardChargerWrapper instead of CardCharger, preventing development error for unit tests.
During code review, you can find whether CardCharger being used in other class instead of CardChargerWrapper. This is to ensure no leak of code.
I'm unsure, but seems like you can hide references of your main project to your real CardCharger. This will protect your code further.
If [isInUnitTest() is an antipattern], how can I still enforce that unit tests don't reach external dependencies?
I now have a solution I'm fairly satisfied with, which will ensure external dependencies cannot be used in test environments without explicitly enabling them. All classes which depend on external resources (HTTP requests, databases, credit card processors, etc.) take as one of their arguments a configuration class which contains the necessary settings to initialize these objects. In a real environment, a real Config object is passed in containing the data they need. In a test environment, a mock is passed in, and without explicitly configuring the mock, the object will fail to construct/connect.
For instance, I have an Http connection utility:
public class HttpOp {
public HttpOp(Config conf) {
if(!conf.isHttpAllowed()) {
throw new UnsupportedOperationException("Cannot execute HTTP requests in "+
getClass()+" if HTTP is disabled. Did you mean to mock this class?");
}
}
....
}
In a unit test, if you attempted to run code that constructs an HttpOp object, you'd raise an exception as the mocked Config will not return true unless explicitly set to do so. In a functional test where this is desired, you can do so explicitly:
#Test
public void connect() {
State.Http httpState = mock(State.Http.class);
when(httpState.isEnabled()).thenReturn(true);
RemoteDataProcessor rdp = new RemoteDataProcessor(new HttpOp(httpState));
...
}
Of course, this still depends on Config being properly mocked in test environments, but now we have exactly one danger point to look for and reviewers can quickly verify the Config object is mocked and trust that only explicitly enabled utilities will be accessible. Similarly, there's now only one gotcha new team-members need to be told ("always mock Config in tests") and they can now be confident they won't accidentally charge a credit card or send emails to clients.
Update
This was an idea I had shortly after posting this question, however I'm now convinced it's not a good plan. Leaving it here for posterity, but see my newer answer for what I ended up doing.
I'm far from certain this is the right thing to do, but one thought I had, which at least addresses my first and third objections, comes from Determine if code is running as part of a unit test:
You could avoid storing external state about whether you're in a unit test or not by directly examining the execution stack, like so:
/**
* Determines at runtime and caches per-thread if we're in unit tests.
*/
public static class IsInUnitTest extends ThreadLocal<Boolean> {
private static final ImmutableSet<String> TEST_PACKAGES =
ImmutableSet.of("org.testng");
#Override
protected Boolean initialValue() {
for(StackTraceElement ste : Thread.currentThread().getStackTrace()) {
for(String pkg : TEST_PACKAGES) {
if(ste.getClassName().startsWith(pkg)) {
return true;
}
}
}
return false;
}
}
The primary advantage here is we don't store any state; we simply check the stack trace - if the trace contains a test-framework package, we're in a unit test, otherwise we're not. It's not perfect - in particular it could potentially lead to false positives if you use the same testing framework for integration or other more permissive tests - but avoiding external state seems like at least a slight win.
Curious what others think of this idea.
I'd like to brush my brain to avoid confusions. In few words, what can be said about Mocking process in TDD
What's the GREAT idea behind MOCKING?
Mocking frameworks are meant to be used only to avoid accessing DB during tests or they can be used for something else?
For new comers (like me), are all the frameworks equal or I need to choose one for this or that reason?
In addition to eliminating databases and other slow or ancillary concerns from the unit being tested, mocking allows you to start writing tests for a class without having to implement any collaborating classes.
As you design some piece of functionality, you'll realize that you need some other class or service, in order to stick to the single responsibility principle, but then you'll have to implement those to get the first one working, which in turn will demonstrate the need for still more classes.
If you can mock or stub those dependencies, then you can create the interfaces upon which that first class will rely, without actually having to implement anything outside of that class -- just return canned results from stubs of the interfaces.
This is an essential component to a test-first approach.
The GREAT idea: LIMIT THE SCOPE OF YOUR TESTS. By removing dependencies you remove the risk of test failures because of dependencies. That way you can focus on the correctness of the code that USES those dependencies.
Mocking DB's is very common but you can mock any dependency with an interface. In a recent project we mocked a web service, for example. You might even want to mock another business object just to make sure that you aren't relying on the correctness of the logic in that object.
I'd choose whichever one seems easiest to use. Moq is really nice.
I suggest you start here:
Mocks are not Stubs
It probably is the article that got me thinking the right way about Mocks. Sure the mocked object is usually heavy (otherwise it may not be worth mocking) but it doesn't have to be heavy in the sense that has some strong reliance on an external system like a database. It can be just a complex piece that you need to isolate to effectively be testing only your class and not the dependency.
I have read this: http://martinfowler.com/articles/mocksArentStubs.html
My concepts about a stub and a mock are clear. I understand the need of isolation frameworks like moq, rhinomocks and like to create a mock object. As mocks, participate in actual verfication of expectations. But why do we need these frameworks to create stubs. I would rather prefer rolling out a hand created stub and use it in various fixtures.
Have you tried using a library like Rhino Mocks for a day or two before deciding that hand-rolled stubs is a better option?
#chibacity: The whole idea about mocking libraries is to avoid implementations. If I only need to set a single property's value, I don't want to create a whole implementation of an interface. Calling code like
MyObj obj = MockRepository.GenerateStub<MyObj>();
obj.MyProperty = 33;
seems much simpler to me. Not to mention situations where I only need a dumb stub to be passed as a parameter where I don't care what happens to it (so no set up is required).
This doesn't mean you shouldn't roll out your own stubs for more complex scenarios, but in my experience such cases are rare.
My recommendation is to learn how to use a good mocking library (like Rhino) with all its little tricks, my bet is that you'll soon learn to appreciate the reasons for its existence.
Strictly speaking, no, you do not need an isolation framework to create stubs. In fact, Microsoft even has a Stubs Framework that only generates stubs, not mocks.
I've never written an isolation framework myself, but it would seem that it once you have object mocking in place, stubs would be a breeze to create. That might be the main reason that most/all isolation frameworks include stub objects.
In regards to your last sentence ("I would rather prefer rolling out a hand created stub and use it in various fixtures"), have you actually tried that on any sizable project? Sure, maybe you have an interface with a single method that returns a nullable bool -- you only have to write three stubs for that interface, and that's not so bad.
But once you start looking at dozens of interfaces and classes to stub, it simply becomes a mess to keep track of all the different stubs. If you are using each stub in several tests, you can certainly justify hand-writing a stub and putting it aside; but when you only use one particular stub once or twice, it's much easier to keep it as an "anonymous" stub directly generated by the framework, for simplicity's sake.
After using mocking frameworks for a while, I have found that my code design takes on a totally different slant. This slant seems to be directed more in an interaction style. In other words, I become more interested in messages and behavior than state. My objects become services rather than stateful objects as would happen when using stubs. With stubs I end up passing state around to other objects.
The problem for me then becomes more of creating abstraction layers. I have to question whither something should be interacting at certain levels. This helps with creating that 'Last responsible Moment'. So I end up with objects that have producers and consumers. And everything in between is just message channels.
I hope this helps.
I recently discussed with a colleague about mocking. He said that mocking classes is very bad and should not be done, only in few cases.
He says that only interfaces should be mocked, otherwise it's an architecture fault.
I wonder why this statement (I fully trust him) is so correct? I don't know it and would like to be convinced.
Did I miss the point of mocking (yes, I read Martin Fowler's article)
Mocking is used for protocol testing - it tests how you'll use an API, and how you'll react when the API reacts accordingly.
Ideally (in many cases at least), that API should be specified as an interface rather than a class - an interface defines a protocol, a class defines at least part of an implementation.
On a practical note, mocking frameworks tend to have limitations around mocking classes.
In my experience, mocking is somewhat overused - often you're not really interested in the exact interaction, you really want a stub... but mocking framework can be used to create stubs, and you fall into the trap of creating brittle tests by mocking instead of stubbing. It's a hard balance to get right though.
IMHO, what your colleague means is that you should program to an interface, not an implementation. If you find yourself mocking classes too often, it's a sign you broke the previous principle when designing your architecture.
Mocking classes (in contrast to mocking interfaces) is bad because the mock still has a real class in the background, it is inherited from, and it is possible that real implementation is executed during the test.
When you mock (or stub or whatever) an interface, there is no risk of having code executed you actually wanted to mock.
Mocking classes also forces you to make everything, that could possibly be mocked, to be virtual, which is very intrusive and could lead to bad class design.
If you want to decouple classes, they should not know each other, this is the reason why it makes sense to mock (or stub or whatever) one of them. So implementing against interfaces is recommended anyway, but this is mentioned here by others enough.
I would suggest to stay away from mocking frameworks as far as possible. At the same time, I would recommend to use mock/fake objects for testing, as much as possible. The trick here is that you should create built-in fake objects together with real objects. I explain it more in detail in a blog post I wrote about it: http://www.yegor256.com/2014/09/23/built-in-fake-objects.html
Generally you'd want to mock an interface.
While it is possible to mock a regular class, it tends to influence your class design too much for testability. Concerns like accessibility, whether or not a method is virtual, etc. will all be determined by the ability to mock the class, rather than true OO concerns.
There is one faking library called TypeMock Isolator that allows you to get around these limitations (have cake, eat cake) but it's pretty expensive. Better to design for testability.
The answer, like most questions about practices, is "it depends".
Overuse of mocks can lead to tests that don't really test anything. It can also lead to tests which are virtual re-implementations of the code under test, tightly bound to a specific implementation.
On the other hand, judicious use of mocks and stubs can lead to unit tests which are neatly isolated and test one thing and one thing alone - which is a good thing.
It's all about moderation.
It makes sense to mock classes so tests can be written early in the development lifecycle.
There is a tendency to continue to use mock classes even when concrete implementations become available. There is also the tendency to develop against mock classes (and stubs) necessary early in a project when some parts of the system have not been built.
Once a piece of the system has been built it is necessary to test against it and continue to test against it (for regression). In this case starting with mocks is good but they should be discarded in favour of the implementation as soon as possible. I have seen projects struggle because different teams continue to develop against the behaviour of the mock rather than the implementation (once it is available).
By testing against mocks you are assuming that the mock is characteristic of the system. Often this involves guessing what the mocked component will do. If you have a specification of the system you are mocking then you don't have to guess, but often the 'as-built' system doesn't match the original specification due to practical considerations discovered during construction. Agile development projects assume this will always happen.
You then develop code that works with the mock. When it turns out that the mock does not truly represent the behaviour of the real as-built system (eg. latency issues not seen in the mock, resource and efficiency issues not seen in the mock, concurrency issues, performance issues etc) you then have a bunch of worthless mocking tests you must now maintain.
I consider the use of mocks to be valuable at the start of development but these mocks should not contribute to project coverage. It is best later if the mocks are removed and proper integration tests are created to replace them otherwise your system will not be getting tested for the variety of conditions which your mock did not simulate (or simulates incorrectly relative to the real system).
So, the question is whether or not to use mocks, it is a matter of when to use them and when to remove them.
It depends how often you use (or are forced by bad design) mocks.
If instantiating the object becomes too hard (and it happens more than often), then it is a sign the code may need some serious refactoring or change in design (builder? factory?).
When you mock everything you end up with tests that know everything about your implementation (white box testing). Your tests no longer document how to use the system - they are basically a mirror of its implementation.
And then comes potential code refactoring..
From my experience it's one of the biggest issues related to overmocking. It becomes painful and takes time, lots of it.
Some developers become fearful of refactoring their code knowing how long will it take.
There is also question of purpose - if everything is mocked, are we really testing the production code?
Mocks of course tend to violate DRY principle by duplicating code in two places: once in the production code and once in the tests.
Therefore, as I mentioned before, any change to code has to be made in two places (if tests aren't written well, it can be in more than that..).
Edit: Since you have clarified that your colleague meant mock class is bad but mock interface is not, the answer below is outdated. You should refer to this answer.
I am talking about mock and stub as defined by Martin Fowler, and I assume that's what your colleague meant, too.
Mocking is bad because it can lead to overspecification of tests. Use stub if possible and avoid mock.
Here's the diff between mock and stub (from the above article):
We can then use state verification on
the stub like this.
class OrderStateTester...
public void testOrderSendsMailIfUnfilled() {
Order order = new Order(TALISKER, 51);
MailServiceStub mailer = new MailServiceStub();
order.setMailer(mailer);
order.fill(warehouse);
assertEquals(1, mailer.numberSent());
}
Of course this is a very simple test -
only that a message has been sent.
We've not tested it was send to the
right person, or with the right
contents, but it will do to illustrate
the point.
Using mocks this test would look quite
different.
class OrderInteractionTester...
public void testOrderSendsMailIfUnfilled() {
Order order = new Order(TALISKER, 51);
Mock warehouse = mock(Warehouse.class);
Mock mailer = mock(MailService.class);
order.setMailer((MailService) mailer.proxy());
mailer.expects(once()).method("send");
warehouse.expects(once()).method("hasInventory")
.withAnyArguments()
.will(returnValue(false));
order.fill((Warehouse) warehouse.proxy());
}
}
In order to use state verification on the stub, I need to make some extra methods on the >stub to help with verification. As a result the stub implements MailService but adds extra >test methods.
I have read a few questions on StackOverflow that ask how functions such as time.Now() can be mocked. The solution seems to be to write a struct that implements a particular interface, and then in the tests, pass in that mock instead.
Is there a better way of doing this? Since there is no good enough implementation of a dependency injection container in Golang, I don't want to pass structs around manually, or to create a struct that only has something like this inside:
func (s MyStruct) Now() {
return time.Now()
}
Because, first, I can't test this function. It might be nothing when you are dealing with one line that has no variables in it, but I have a function that connects to a TCP port. Doing stuff like this would require me to create adapters to everything it uses (eg net.Dial(), time.Now(), io.Copy(), bufio.Read(), and many others).
Second, since there is no DI, I will have to pass structs around, which will end up cluttering my code, rather than making it easier to read like DI does in other languages.
Besides enabling you to easily mock these functions, writing 'adapters' (your own interface) for such things has the advantage of abstracting your code from any individual implementation.
Passing structs around in your code also keeps your app from external dependencies on some kind of injection framework (many of which just violate the principle of IOC anyway).
In addition to writing your own interfaces to these dependencies, certain design philosophies can help you to mock .
For example, using domain driven design patterns, you can simply mock your "repository" in unit tests rather than mocking a database library.
In Go, all of this ends up very clean, and succinct, without the burden of a horde of third party frameworks magically doing the work for you.