I've been reading that static methods tend to be avoided when using TDD because they tend to be hard to mock. I find though, that the easiest thing to unit test is a static method that has simple functionality. Don't have to instantiate any classes, encourages methods that a simple, do one thing, are "standalone" etc.
Can someone explain this discrepancy between TDD best practices and pragmatic ease?
thanks,
A
A static method is easy to test, but something that directly calls a static method generally is not easy to test independent of the static method it depends on. With a non-static method you can use a stub/mock/fake instance to ease testing, but if the code you're testing calls static methods it's effectively "hard-wired" to that static method.
The answer to the asked question is, in my opinion "Object Oriented seems to be all that TDD people think about."
Why? I don't know. Maybe they are all Java programmers who've been infected with the disease of making everything rely on six indirection layers, dependency injection and interface adapters.
Java programmers seem to love to make everything difficult up front in order to "save time later."
I advise applying some Agile principles to your TDD: If it isn't causing a problem then don't fix it. Don't over design.
In practice I find that if the static methods are tested well first then they are not going to be the cause of bugs in their callers.
If the static methods execute quickly then they don't need a mock.
If the static methods work with stuff from outside the program, then you might need a mock method. In this case you'd need to be able to simulate many different kinds of function behavior.
If you do need to mock a static method remember that there are ways to do it outside of OO programming.
For example, you can write scripts to process your source code into a test form that calls your mock function. You could link different object files that have different versions of the function into the test programs. You could use linker tricks to override the function definition (if it didn't get inlined). I am sure there are some more tricks I haven't listed here.
It's easy to test the static method. The problem is that there is no way to isolate your other code from that static method when testing the other code. The calling code is tightly-coupled to the static code.
A reference to a static method cannot be mocked by many mocking frameworks nor can it be overridden.
If you have a class that is making lots of static calls, then to test it you have to configure the global state of the application for all of those static calls - so maintenance becomes a nightmare. And if your test fails, then you don't know which bit of code caused the failure.
Getting this wrong, is one of the reasons that many developers think TDD is nonsense. They put in a huge maintenance effort for test results that only vaguely indicate what went wrong. If they'd only reduced the coupling between their units of code, then maintenance would be trivial and the test results specific.
That advice is true for the most part.. but not always. My comments are not C++ specific..
writing tests for static methods (which are pure/stateless functions): i.e. the work off the inputs to produce a consistent result. e.g. Add below - will always give the same value given a particular set of inputs. There is no problem in writing tests for these or code that calls such pure static methods.
writing tests for static methods which consume static state : e.g. GetAddCount() below. Calling it in multiple tests can yield different values. Hence one test can potentially harm the execution of another test - tests need to be independent. So now we need to introduce a method to reset static state such that each test can start from a clean slate (e.g. something like ResetCount()).
Writing tests for code that accesses static methods but no source-code access to the dependency: Once again depends on the properties of the static methods themselves. However if they are gnarly, you have a difficult dependency. If the dependency is an object, then you could add a setter to the dependent type and set/inject a fake object for your tests. When the dependency is static, you may need a sizable refactoring before you can get tests running reliably. (e.g. Add an object middle-man dependency that delegates to the static method. Now plugin a fake middle-man for your tests)
Lets take an example
public class MyStaticClass
{
static int __count = 0;
public static int GetAddCount()
{ return ++__count; }
public static int Add(int operand1, int operand2)
{ return operand1 + operand2; }
// needed for testability
internal static void ResetCount()
{
__count = 0;
}
}
...
//test1
MyStaticClass.Add(2,3); // => 5
MyStaticClass.GetAddCount(); // => 1
// test2
MyStaticClass.Add(2,3); // => 5
//MyStaticClass.ResetCount(); // needed for tests
MyStaticClass.GetAddCount(); // => unless Reset is done, it can differ from 1
Related
I have a class that looks similar to this:
public class DiscoveryDocumentCache
{
static DiscoveryDocumentCache()
{
cache = new MemoryCache(discoveryDocumentCacheName);
}
private static MemoryCache cache;
// Other stuff...
}
I want the memory cache to be static so that it is shared across all instances of the class.
It is working great for the actual use cases. Where I am struggling is with my unit tests. Because the cache variable is static, it accumulates all the different values that get put in it.
My use case does not have a "remove" option. (They just expire out of the cache.)
I have added this to my DiscoveryDocumentCache class:
public void __UnitTesting__CleanUpMemoryCacheBetweenTests()
{
cache.Dispose();
cache = new MemoryCache(discoveryDocumentCacheName);
}
But it feels dirty to add a method that is just to facilitate unit testing.
.NET Framework had some ways to access class's private variables if you really thought you should. Is there something similar I can do to reset this static variable in my TearDown unit test method?
Not sure if you'll think this is an answer, or just a long comment. :-)
Your tests are working well because they are telling you something. They're saying "This is a bad use of a static instance."
Statics are always suspicious, although not always bad. What's bad here is you use a static to implement the singleton pattern in a way that is subsequently out of your control. That is, your individual instances are deciding for themselves which cache (the singleton) to use, rather than being told what to use.
As #Vaccano suggests, you are better off to use some sort of dependency injection. Translate that to "Tell each object what cache to use rather than having them figure it out for itself." This is one of those things that requires a bit of up-front investment in order to avoid saving trouble later.
As an aside... there is nothing wrong with changing your application to make it more testable. Testability is just another of those "ilities" that define good software design. The trick is to change it so as to improve the software, usually simplifying it rather than complicating it.
Consider you have the following method:
public Foo ParseMe(string filepath)
{
// break up filename
// validate filename & extension
// retrieve info from file if it's a certain type
// some other general things you could do, etc
var myInfo = GetFooInfo(filename);
// create new object based on this data returned AND data in this method
}
Currently I have unit tests for GetFooInfo, but I think I also need to build unit tests for ParseMe. In a situation like this where you have a two methods that return two different properties - and a change in either of them could break something - should unit tests be created for both to determine the output is as expected?
I like to err on the side of caution and be more wary about things breaking and ensuring that maintenance later on down the road is easier, but I feel very skeptical about adding very similar tests in the test project. Would this be bad practice or is there any way to do this more efficiently?
I'm marking this as language agnostic, but just in case it matters I am using C# and NUnit - Also, I saw a post similar to this in title only, but the question is different. Sorry if this has already been asked.
ParseMe looks sufficiently non-trivial to require a unit test. To answer your precise question, if "you have a two methods that return two different properties - and a change in either of them could break something" you should absolutely unit test them.
Even if the bulk of the work is in GetFooInfo, at minimum you should test that it's actually called. I know nothing about NUnit, but I know in other frameworks (like RSpec) you can write tests like GetFooInfo.should be_called(:once).
It is not a bad practice to test a method that is calling another method. In fact, it is a good practice. If you have a method calling another method, it is probably performing additional functionality, which should be tested.
If you find yourself unit testing a method that calls a method that is also being unit tested, then you are probably experiencing code reuse, which is a good thing.
I agree with #tsm - absolutely test both methods (assuming both are public).
This may be a smell that the method or class is doing too much - violating the Single Responsibility Principle. Consider doing an Extract Class refactoring and decoupling the two classes (possibly with Dependency Injection). That way you could test both pieces of functionality independently. (That said, I'd only do that if the functionality was sufficiently complex to warrant it. It's a judgment call.)
Here's an example in C#:
public interface IFooFileInfoProvider
{
FooInfo GetFooInfo(string filename);
}
public class Parser
{
private readonly IFooFileInfoProvider _fooFileInfoProvider;
public Parser(IFooFileInfoProvider fooFileInfoProvider)
{
// Add a null check
_fooFileInfoProvider = fooFileInfoProvider;
}
public Foo ParseMe(string filepath)
{
string filename = Path.GetFileName(filepath);
var myInfo = _fooFileInfoProvider.GetFooInfo(filename);
return new Foo(myInfo);
}
}
public class FooFileInfoProvider : IFooFileInfoProvider
{
public FooInfo GetFooInfo(string filename)
{
// Do I/O
return new FooInfo(); // parameters...
}
}
Many developers, me included, take a programming by contract approach. That requires you to consider each method as a black box. If the method delegates to another method to accomplish its task does not matter, when you are testing the method. But you should also test all large or complicated parts of your program as units. So whether you need to unit test the GetFooInfo depends on how complicated that method is.
I have this CacheManager class which keeps a static dictionary with all sorts of cached data. However, with this dictionary being static it gets filled up with data from the other unit tests. This keeps me from unit testing whether or not the CacheManager is empty on init, and breaks with the principles of unit testing.
Any ideas how to create a proper unit test for this?
Code
public class CacheManager
{
private static readonly Dictionary<ICacheKey, ListCacheItem> cacheEntries =
new Dictionary<ICacheKey, ListCacheItem>();
public static Dictionary<ICacheKey, ListCacheItem> CacheEntries
{
get
{
lock (cacheEntries)
{
return cacheEntries;
}
}
}
Generally, this is not a good idea from a testing perspective. By making the members of CacheManager static, you will never be able to isolate it in such a way to make it nice to unit test.
Perhaps a better solution is the Singleton Pattern. To do this, get rid of the static modifiers on CacheManager's members. Then you can have one static instance in your app that is used by everyone else. Therefore, in your unit test, you could create a new instance of the class that you can test in isolation, but still have the desired functionality.
Short answer: you can not do it properly. Unit testing and statics do not play really well together, you will (almost) always run into problems like the one you mentioned.
Longer answer: best solution would be to refactor your code. Even if you need the singleton behavior you have several options (e.g. dependency injection). David's recommendation is of course also an option that would at least let you test your cache, but you may still have problems when you want to test the rest of the system.
If for some reason you want to stick to your current design you can still have some (not necessary nice) workarounds. Some examples:
Easiest might be to add a "cleanCache" method. In some situations it might be even useful for the rest of the system, and each of your tests could also do it as the first step (in "setup/beforeTest or similar methods").
You could also play with visibility and let your tests do cleanup that is not allowed for the rest of the code.
These hacks will probably work as long as you do not run your tests in parallel.
So I have a factory class and I'm trying to work out what the unit tests should do. From this question I could verify that the interface returned is of a particular concrete type that I would expect.
What should I check for if the factory is returning concrete types (because there is no need - at the moment - for interfaces to be used)? Currently I'm doing something like the following:
[Test]
public void CreateSomeClassWithDependencies()
{
// m_factory is instantiated in the SetUp method
var someClass = m_factory.CreateSomeClassWithDependencies();
Assert.IsNotNull(someClass);
}
The problem with this is that the Assert.IsNotNull seems somewhat redundant.
Also, my factory method might be setting up the dependencies of that particular class like so:
public SomeClass CreateSomeClassWithDependencies()
{
return new SomeClass(CreateADependency(), CreateAnotherDependency(),
CreateAThirdDependency());
}
And I want to make sure that my factory method sets up all these dependencies correctly. Is there no other way to do this then to make those dependencies public/internal properties which I then check for in the unit test? (I'm not a big fan of modifying the test subjects to suit the testing)
Edit: In response to Robert Harvey's question, I'm using NUnit as my unit testing framework (but I wouldn't have thought that it would make too much of a difference)
Often, there's nothing wrong with creating public properties that can be used for state-based testing. Yes: It's code you created to enable a test scenario, but does it hurt your API? Is it conceivable that other clients would find the same property useful later on?
There's a fine line between test-specific code and Test-Driven Design. We shouldn't introduce code that has no other potential than to satisfy a testing requirement, but it's quite alright to introduce new code that follow generally accepted design principles. We let the testing drive our design - that's why we call it TDD :)
Adding one or more properties to a class to give the user a better possibility of inspecting that class is, in my opinion, often a reasonable thing to do, so I don't think you should dismiss introducing such properties.
Apart from that, I second nader's answer :)
If the factory is returning concrete types, and you're guaranteeing that your factory always returns a concrete type, and not null, then no, there isn't too much value in the test. It does allows you to make sure, over time that this expectation isn't violated, and things like exceptions aren't thrown.
This style of test simply makes sure that, as you make changes in the future, your factory behaviour won't change without you knowing.
If your language supports it, for your dependencies, you can use reflection. This isn't always the easiest to maintain, and couples your tests very tightly to your implementation. You have to decide if that's acceptable. This approach tends to be very brittle.
But you really seem to be trying to separate which classes are constructed, from how the constructors are called. You might just be better off with using a DI framework to get that kind of flexibility.
By new-ing up all your types as you need them, you don't give yourself many seams (a seam is a place where you can alter behaviour in your program without editing in that place) to work with.
With the example as you give it though, you could derive a class from the factory. Then override / mock CreateADependency(), CreateAnotherDependency() and CreateAThirdDependency(). Now when you call CreateSomeClassWithDependencies(), you are able to sense whether or not the correct dependencies were created.
Note: the definition of "seam" comes from Michael Feather's book, "Working Effectively with Legacy Code". It contains examples of many techniques to add testability to untested code. You may find it very useful.
What we do is create the dependancies with factories, and we use a dependancy injection framework to substitute mock factories for the real ones when the test is run. Then we set up the appropriate expectations on those mock factories.
You can always check stuff with reflection. There is no need to expose something just for unit tests. I find it quite rare that I need to reach in with reflection and it may be a sign of bad design.
Looking at your sample code, yes the Assert not null seems redundant, depending on the way you designed your factory, some will return null objects from the factory as opposed to exceptioning out.
As I understand it you want to test that the dependencies are built correctly and passed to the new instance?
If I was not able to use a framework like google guice, I would probably do it something like this (here using JMock and Hamcrest):
#Test
public void CreateSomeClassWithDependencies()
{
dependencyFactory = context.mock(DependencyFactory.class);
classAFactory = context.mock(ClassAFactory.class);
myDependency0 = context.mock(MyDependency0.class);
myDependency1 = context.mock(MyDependency1.class);
myDependency2 = context.mock(MyDependency2.class);
myClassA = context.mock(ClassA.class);
context.checking(new Expectations(){{
oneOf(dependencyFactory).createDependency0(); will(returnValue(myDependency0));
oneOf(dependencyFactory).createDependency1(); will(returnValue(myDependency1));
oneOf(dependencyFactory).createDependency2(); will(returnValue(myDependency2));
oneOf(classAFactory).createClassA(myDependency0, myDependency1, myDependency2);
will(returnValue(myClassA));
}});
builder = new ClassABuilder(dependencyFactory, classAFactory);
assertThat(builder.make(), equalTo(myClassA));
}
(if you cannot mock ClassA you can assign a non-mock version to myClassA using new)
I have started to write my tests as static functions in the class I want to test. These functions usually test functionality, they create a lot of objects, measure memory consumption (to detect leaks), etc.
Is this good or bad practice? Will it bite me in the long run?
I keep them separate, because I don't want test classes to be included in the deployable artifacts. No sense increasing the size of the .exe or making them available to clients.
I'd recommend writing unit tests with CppUnit.
No, you should write unit tests instead.
I would say that it isn't best-practice, but it is "okay", in that it won't break your program or cause it to crash. There are a lot of things that are okay to do, but you shouldn't do.
Test code is fundamentally different from production code in terms of ownership, deployment, non-functional requirements and so on. Therefore it is better to keep it separate from the code being tested, in separate files and probably even in separate directories.
To facilitate whitebox unit testing for the class under test, you often need to declare the test class/test functions as friend. Some classes are unit-testable with the public members only so adding friends is not always necessary.
Combining test code and code under test is simple: you just link the object files together in the same project.
Sometimes you can see unit test code that #includes the code under test but I would advice against that - for example, if you have test coverage measurement tooling in place (highly recommended!), the measures won't be correct for the code under test.
If you have your test cases inside your class, it's hard to have things like fixtures.
I am also going to give a shout out to Boost.Test. The learning curve is a little high but it is amazing once you get used to it.
That is bad practice.
You should have a separate class to test the class you are creating. The way you are doing you are bloating production code with test code. This is not what you should do.
The way you want to test the class Foo is like this:
//Foo.cpp
class Foo {
public:
int GetInt() { return 15; }
};
//FooTest.cpp
TEST(FooTest, testGetIntShouldReturn15) {
Foo foo;
ASSERT_EQUAL(15, foo.GetInt());
}