I have a class that looks similar to this:
public class DiscoveryDocumentCache
{
static DiscoveryDocumentCache()
{
cache = new MemoryCache(discoveryDocumentCacheName);
}
private static MemoryCache cache;
// Other stuff...
}
I want the memory cache to be static so that it is shared across all instances of the class.
It is working great for the actual use cases. Where I am struggling is with my unit tests. Because the cache variable is static, it accumulates all the different values that get put in it.
My use case does not have a "remove" option. (They just expire out of the cache.)
I have added this to my DiscoveryDocumentCache class:
public void __UnitTesting__CleanUpMemoryCacheBetweenTests()
{
cache.Dispose();
cache = new MemoryCache(discoveryDocumentCacheName);
}
But it feels dirty to add a method that is just to facilitate unit testing.
.NET Framework had some ways to access class's private variables if you really thought you should. Is there something similar I can do to reset this static variable in my TearDown unit test method?
Not sure if you'll think this is an answer, or just a long comment. :-)
Your tests are working well because they are telling you something. They're saying "This is a bad use of a static instance."
Statics are always suspicious, although not always bad. What's bad here is you use a static to implement the singleton pattern in a way that is subsequently out of your control. That is, your individual instances are deciding for themselves which cache (the singleton) to use, rather than being told what to use.
As #Vaccano suggests, you are better off to use some sort of dependency injection. Translate that to "Tell each object what cache to use rather than having them figure it out for itself." This is one of those things that requires a bit of up-front investment in order to avoid saving trouble later.
As an aside... there is nothing wrong with changing your application to make it more testable. Testability is just another of those "ilities" that define good software design. The trick is to change it so as to improve the software, usually simplifying it rather than complicating it.
Related
I like unit testing, it is proving its worth immensely for the last year and a half or so ive used it. However I always have a problem or rather a niggle with private methods (and protected).
I don't want to make them public or use internals visible to attribute. I want a clean and crisp solution - that is testable and i'd be proud to let someone else look at.
I am coming to the conclusion that if a private method really needs testing independantly then maybe it should be moved out onto another interface and use association to expose the functionality to the calling method. I believe this in essence to be the Facade pattern.
Is this really the best way to go about this?
Or more objectively ... are there any other approaches I have totally overlooked?
Edit: Are we talking about a specific language?
I am working in C#. I had kept code out of the question as i was looking for something abstract. Coming back to it today i realise that is perhaps folly due to languages really being that different.
So some code:
public class CopmlexClass
{
public void SomeMethod()
{ }
private void workerMethod()
{ }
}
would get re factored into
public class CopmlexClass
{
public void SomeMethod()
{ }
public IComplexClassWorker Worker { get; set; }
}
public interface IComplexClassWorker
{
void WorkerMethod();
}
In fact id probably prefer to use constructor injection and not even expose the property
My question is: is that the best way? what are the alternatives bar reflection / internals visible to attribute?
A private method which needs to be tested independently can be the result of the following:
your class is doing too much - its public methods implement functionality which is too complex to be unit tested as a whole, and/or
the effects of calling the specific private method can't be directly sensed from outside the class.
Both cases are usually a clear call to extract another class containing some private method(s) of the original class, turned into public, thus made directly testable. (The sometimes challenging part is to find logically cohesive chunks of functionality which can form useful classes on their own right. You may not always get a perfect result at first - in refactoring, sometimes one needs to make a compromise, a small step into some not-yet-clearly-defined direction. In the long term, such a step may open up new possibilities, call attention to other similar code parts, or the new class may start to attract code bits from other places, eventually forming a coherent class. Or turning into something completely new, but better than what you originally envisioned.)
Exposing the private method via another interface / facade is IMO not going to solve the problem in the long term, only muddles the waters. Classes should have a well defined, complete and minimal interface. Exposing private methods in any way may open up ways to compromise the internal state of the object, which is a Bad Thing.
When we started writing unit tests in our team a couple of years ago we started with the rules you set out above - i.e. we test the public interface of an assembly.
We expected one advantage to be in detecting unreachable code. If the code coverage tools detect code blocks which not being tested, then either tests are missing or the code is unreachable and should be removed.
But in practice we haven't stuck to them. We have a very modular design - more than 30 projects in our main solution (most having a matching unit tests project). We now usually give the test project access to the internals of the project under test.
I think one problem is that we are not automatically using code coverage to detect missing tests or unreachable code. Because this is a manual process, it doesn't get done.
I am wondering if unit testing private methods is a good practice?
Normally only public interface should be tested.
However, I have found out that during complex calculation, which calls tons of different private methods, it is easier to unit test the private methods first, and then make a simple test for the public interface method.
As an example let's say you have an audio player and you have functions:
void play(){ ... }
void pause(){ ... }
void seek(time t)
{
//All Private methods
checkIfValidTimeRange(...);
moveToFilePos(...);
fillBuffers(...);
}
Normally I would write unit tests for : checkIfValidTimeRange(...), moveToFilePos(...), fillBuffers(...).
But I am not sure if doing so is good practice.
It's not a good practice (yet that doesn't mean you should never do that), and if possible you want to avoid it. Testing private method usually means your design could be better. Let's take a quick look at your player example:
moveToFilePos: sounds more like a responsibility of something doing I\O operations, not a music player's
fillBuffers: more of a memory manager's job rather than music player
checkIfValidTimeRange: again, probably could be moved out of player's scope to some simple validation class (seems like this one might be useful in other places aswell)
At the moment your music player does I/O, memory management and what not else. Is that all really in scope of its responsibilities?
If your private methods are complex enough to warrant testing, you're likely missing some classes where those private methods are turned public.
You certainly can test private methods, but you should take the need to do it as a hint there's something wrong in your design.
IMHO it's a very good idea, I do it all the times. I usually create a helper class which makes the private methods accessable and test it..
Usually it's even easier to test private methods, since they do something very specific. On the other hand you might have a big public method which is a bit harder to test. So it certainly simplifies unit tests.
Which part of your code base is your private methods relying on ?
If somebody changes the way one of the method you are relying on works and thus breaks your method, isn't it worth knowing it ?
Testing is not only for checking that your method behaves as it should, but also to check that changes in other parts of the codebase doesn't break your method.
So unless your method is only using basic constructs of your language, test it !
I've been reading that static methods tend to be avoided when using TDD because they tend to be hard to mock. I find though, that the easiest thing to unit test is a static method that has simple functionality. Don't have to instantiate any classes, encourages methods that a simple, do one thing, are "standalone" etc.
Can someone explain this discrepancy between TDD best practices and pragmatic ease?
thanks,
A
A static method is easy to test, but something that directly calls a static method generally is not easy to test independent of the static method it depends on. With a non-static method you can use a stub/mock/fake instance to ease testing, but if the code you're testing calls static methods it's effectively "hard-wired" to that static method.
The answer to the asked question is, in my opinion "Object Oriented seems to be all that TDD people think about."
Why? I don't know. Maybe they are all Java programmers who've been infected with the disease of making everything rely on six indirection layers, dependency injection and interface adapters.
Java programmers seem to love to make everything difficult up front in order to "save time later."
I advise applying some Agile principles to your TDD: If it isn't causing a problem then don't fix it. Don't over design.
In practice I find that if the static methods are tested well first then they are not going to be the cause of bugs in their callers.
If the static methods execute quickly then they don't need a mock.
If the static methods work with stuff from outside the program, then you might need a mock method. In this case you'd need to be able to simulate many different kinds of function behavior.
If you do need to mock a static method remember that there are ways to do it outside of OO programming.
For example, you can write scripts to process your source code into a test form that calls your mock function. You could link different object files that have different versions of the function into the test programs. You could use linker tricks to override the function definition (if it didn't get inlined). I am sure there are some more tricks I haven't listed here.
It's easy to test the static method. The problem is that there is no way to isolate your other code from that static method when testing the other code. The calling code is tightly-coupled to the static code.
A reference to a static method cannot be mocked by many mocking frameworks nor can it be overridden.
If you have a class that is making lots of static calls, then to test it you have to configure the global state of the application for all of those static calls - so maintenance becomes a nightmare. And if your test fails, then you don't know which bit of code caused the failure.
Getting this wrong, is one of the reasons that many developers think TDD is nonsense. They put in a huge maintenance effort for test results that only vaguely indicate what went wrong. If they'd only reduced the coupling between their units of code, then maintenance would be trivial and the test results specific.
That advice is true for the most part.. but not always. My comments are not C++ specific..
writing tests for static methods (which are pure/stateless functions): i.e. the work off the inputs to produce a consistent result. e.g. Add below - will always give the same value given a particular set of inputs. There is no problem in writing tests for these or code that calls such pure static methods.
writing tests for static methods which consume static state : e.g. GetAddCount() below. Calling it in multiple tests can yield different values. Hence one test can potentially harm the execution of another test - tests need to be independent. So now we need to introduce a method to reset static state such that each test can start from a clean slate (e.g. something like ResetCount()).
Writing tests for code that accesses static methods but no source-code access to the dependency: Once again depends on the properties of the static methods themselves. However if they are gnarly, you have a difficult dependency. If the dependency is an object, then you could add a setter to the dependent type and set/inject a fake object for your tests. When the dependency is static, you may need a sizable refactoring before you can get tests running reliably. (e.g. Add an object middle-man dependency that delegates to the static method. Now plugin a fake middle-man for your tests)
Lets take an example
public class MyStaticClass
{
static int __count = 0;
public static int GetAddCount()
{ return ++__count; }
public static int Add(int operand1, int operand2)
{ return operand1 + operand2; }
// needed for testability
internal static void ResetCount()
{
__count = 0;
}
}
...
//test1
MyStaticClass.Add(2,3); // => 5
MyStaticClass.GetAddCount(); // => 1
// test2
MyStaticClass.Add(2,3); // => 5
//MyStaticClass.ResetCount(); // needed for tests
MyStaticClass.GetAddCount(); // => unless Reset is done, it can differ from 1
I have this CacheManager class which keeps a static dictionary with all sorts of cached data. However, with this dictionary being static it gets filled up with data from the other unit tests. This keeps me from unit testing whether or not the CacheManager is empty on init, and breaks with the principles of unit testing.
Any ideas how to create a proper unit test for this?
Code
public class CacheManager
{
private static readonly Dictionary<ICacheKey, ListCacheItem> cacheEntries =
new Dictionary<ICacheKey, ListCacheItem>();
public static Dictionary<ICacheKey, ListCacheItem> CacheEntries
{
get
{
lock (cacheEntries)
{
return cacheEntries;
}
}
}
Generally, this is not a good idea from a testing perspective. By making the members of CacheManager static, you will never be able to isolate it in such a way to make it nice to unit test.
Perhaps a better solution is the Singleton Pattern. To do this, get rid of the static modifiers on CacheManager's members. Then you can have one static instance in your app that is used by everyone else. Therefore, in your unit test, you could create a new instance of the class that you can test in isolation, but still have the desired functionality.
Short answer: you can not do it properly. Unit testing and statics do not play really well together, you will (almost) always run into problems like the one you mentioned.
Longer answer: best solution would be to refactor your code. Even if you need the singleton behavior you have several options (e.g. dependency injection). David's recommendation is of course also an option that would at least let you test your cache, but you may still have problems when you want to test the rest of the system.
If for some reason you want to stick to your current design you can still have some (not necessary nice) workarounds. Some examples:
Easiest might be to add a "cleanCache" method. In some situations it might be even useful for the rest of the system, and each of your tests could also do it as the first step (in "setup/beforeTest or similar methods").
You could also play with visibility and let your tests do cleanup that is not allowed for the rest of the code.
These hacks will probably work as long as you do not run your tests in parallel.
So, I'm starting to write some logic for a simple program (toy game on the side). You have a specific ship (called a setup) that is a ship + modules. You start with an empty setup based off a ship and then add modules to that setup. Ships also have a numbered array of module positions.
var setup = new Setup(ship); // ship is a stub (IShip) defined someplace else
var module = new Mock<IModule>().Object;
setup.AddModule(module, 1); // 1 = which position
So, this is the code in my test method. I now need to assert on this code. Well, I need a getter method right?
Assert.AreEqual(module, setup.GetModule(1));
This might sound really dumb and I'm worrying about nothing, but for some stupid reason I'm concerned with adding a method just to assert that a test passed.
Is this fine and is in fact part of the design process that TDD is pushing out? For instance I know I need an AddModule method because I want to test it, and the fact that this requires a GetModule method to test is simply an evolution of my design via TDD.
Or is this kind of a smell because I don't even know if I'll really need GetModule in my code and it will only be used in a test?
For example, adding a module is going to ultimately affect different stats of a setup (armor, shield, firepower, etc). The thing is those are going to be complex, and I wanted to start with a simple test. But in the end, those are the public attributes I care about -- a setup is defined by its stats, not by a list of modules.
Interesting question. I'm glad to hear you're writing the tests first.
If you let the design manifest itself through the tests, you're more likely to build only the parts you'll need. But is this the best design? Maybe not, but don't let that discourage you -- your add method works!
It may be too early to tell if you'll need the GetModule method later. For now, build up the functionality you need and go green, then slowly refactor it (going from red to green again) to get the design you want.
Part of evolving the design is to start with baby steps like a simple method and then grow into the complex stats (eventually dropping this method and changing the test) when enough supports it. When doing TDD, don't expect that the first test you write is targeting the ideal interface. It is OK to have some messiness that will get dropped as you evolve the design.
That being said, if you see no public purpose to the method, try to limit its visibility as much as is reasonable to the test code. Although even that should eventually go away as you get to build out the rest of the system and have something real to test as a side effect of the set method.
I would be wary of introducing a public method in my class that is only used for testing.
There are various ways how you could test this:
Reflection: The GetModule method is a private method in your class (this could also work if your 'stats' are private) and you can access it in your test method via reflection. This will work well, the only trouble is you will not get any compiler errors if you change the name of the private method or add / delete some variables (but, of course, your test will fail and you will know early)
Inheritance: The GetModule method could be protected (only inheritance visible) and your test class could inherit from the main class. This way your test class gets access to this method, but this is not really exposed to the outside world.
Assert the side-effect: This is where you really think about what it means to add a module to the system. If it is going to affect some 'stats' as you put it, you could write tests which assert that the stats are appropriately modified.