I am reviewing some code where the developer has some classes ClassA and ClassB. They both inherit from ParentClass so both must implement a number of abstract methods (e.g. return5Values(2))
In ClassA the values are all double the previous value: [2,4,8,16,32]
In ClassB the values are all +1 the previous value [2,3,4,5,6]
There are also other constraints such as raising an error if the parameter is negative etc.
Other tests like getting the 3rd value only, also exist etc.
(Obviously these are just fake examples to get my point across)
Now, instead of writing a lot of similar tests for both ClassA and ClassB, what the developer has done is created ParentClassChildTests which contains a some code something like this:
public void testVariablesAreCorrect() {
returnedValues = clazz.return5Values(2)
# Does a bunch of other things as well
# ...
assertEqual(expectedValues, returnedValues)
}
ClassATests now inherits from ParentClassChildTest and must define expectedValues as a class variable.
The expectedValues are used within a few different tests as well, so they aren't being defined just for this single test.
Now when ClassATests and ClassBTests are run, it also runs all the tests inside ParentClassChildTests.
My question is: Is this a good method to avoid a lot of duplicate tests and ensure everything works as expected in child classes? Are there any major issues this can lead to? Or a better way of handling this?
Whilst this is all Java code, my question isn't about any particular testing framework or language but the idea in general of inheriting from a parent class which also has tests in it.
The situation that it is possible and sensible to re-use tests for different implementations of an interface / base class is not very common. The following aspects limit the applicability:
Derived classes have different dependencies, which may require different mocks to be created and to be set up. In such a case, the test methods can not be identical. Even if classA and classB currently do not have dependencies or the same dependencies with (coincidentially) the same setup, this can change over time or the next class classC will have different dependencies.
Each derived class will implement different algorithms. In your case, return5Values performs different algorithms in ClassA and ClassB. Due to the different algorithms, the behaviour of the SUT for the same set up and the same inputs may be different: For example, each algorithm will run into overflows at different points. Even the call return5Values(2) that allows to use a derived test for classA and classB today, could with a potential future classC lead to an overflow scenario with possible exceptions thrown.
The different algorithms implemented in the derived classes will have different potential bugs and different corner cases. That is, the necessary set up and inputs for the SUT will have to be different to stimulate the respective boundaries. For some implementations, testing the call return5Values(2) may simply not bring any benefit while test for other parameters than 2 are necessary.
If you share test methods between the classes and only provide the parameters, it is not the test method which is associated with the tests' intent - each parameter set has its own intent. The intent/scenario, however, should ideally be part of the output of each individual test.
Given all these problems, inheritance of test methods does not seem to be the best approach for re-use here. Instead, it may be more beneficial to have have some common helper functions that can be used by the different derived classes.
Having class hierarchies in tests creates dependencies between them. A UnitTest serves the purpose of testing a Unit in isolation where Unit refers to a certain class. I'd argue that it is ok to have helpers and utils to avoid duplicating very basic functionality.
As much as possible unit tests should allow for quick and independent changes of a certain Unit. Having a commonly enforced structure for all tests increases the amount of work to be done if the implementation of unrelated parts of the application changes.
When it comes to integration testing there will be shared functionality for setting up the infrastructure. So the answer is a very clear it depends. Generally it is favorable to reduce dependencies between tests as much as possible and having a base test that determines the inner workings of a derived test is detrimental to that goal.
Related
Suppose my production code starts off like this:
public class SmtpSender
{
....
}
public void DoCoolThing()
{
var smtpSender = new SmtpSender("mail.blah.com"....);
smtpSender.Send(...)
}
I have a brainwave and decide to unit test this function using a fake SmtpSender and dependency injection:
public interface ISmtpSender
{
...
}
public class SmtpSender : ISmtpSender
{
...
}
public class FakeSmtpSender : ISmtpSender
{
...
}
public void DoCoolThing(ISmtpSender smtpSender)
{
smtpSender.Send(...)
}
Then I think wait, all my production code will always want the real one. So why not make the parameter optional, then fill it in with default SmtpSender if not provided?
public void DoCallThing(ISmtpSender smtpSender = null)
{
smtpSender = smtpSender ?? new SmtpSender("...");
smtpSender.Send(...);
}
This allows the unit test to fake it while leaving the production code unaffected.
Is this an acceptable implementation of dependency injection?
If not, what are the pitfalls?
No, having callers depend on concrete implementations of dependencies is not the best way to do dependency injection. Component implementations should always depend only on component interfaces, not on other component implementations.
If one implementation depends on another,
It gives the caller a completely different kind of responsibility than it would otherwise have, that of configuring itself. The caller will thus be less coherent.
It gives the caller a dependency on a class that it would not otherwise have. That means that tests of the caller can break if the dependency is broken, classes needed by the dependency are loaded when the caller is loaded instead of when they're needed, and in compiled languages there would be a compilation-time dependency. All of these make developing large systems harder.
If more than one caller wants the same dependency, each such caller might use this same trick. Then you would have duplication of the concrete dependency among callers, instead of having the concrete dependency in one place (wherever you're specifying all your injected dependencies).
It prevents you from having a useful kind of circular dependency among implementations. Very occasionally I've needed implementations to depend on one another in a circular way. Pure dependency injection makes that relatively safe: implementations always depend on interfaces, so there isn't a compile-time circularity. (Dependency injection frameworks make it possible to construct all the implementations despite the circularity by interposing proxies.) With a concrete dependency, you'll never be able to do that.
I would address the issue you're trying to address by putting the production choice of implementation in the code or config file or whatever that specifies all of your implementations.
It's a valid approach to use, but needs to be strictly limited to certain scenarios. There are several pitfalls to watch out for.
Pros:
Establishes a seam in your code, allowing different implementations to be used as collaborators
Allows clients to use your code without having to specify a collaborator - particularly if most clients would end up using the exact same collaborator
Cons:
Introduces a hidden collaborator into the system
Tightly couples the SUT to the concrete class
Even though other implementations could be provided, if the concrete class changes it will directly affect the SUT (e.g., the constructor changes)
Can easily tightly couple the SUT to other classes as well
For example: not only do you create the dependency, you create all of its dependencies too
In general, it's something you should only reserve for very lightweight defaults (think Null Objects). Otherwise it's very easy for things to quickly spin out of control.
What you're trying to do is similar to using a default constructor in order to make things a little more convenient. You should determine if the convenience is going to have long term benefits, or if it will ultimately come back to haunt you.
Even though you're not using a default constructor, I think you should consider the parallels. Mark Seemann suggests that default constructors could be a design smell: (emphasis mine)
Default constructors are code smells. There you have it. That probably sounds outrageous, but consider this: object-orientation is about encapsulating behavior and data into cohesive pieces of code (classes). Encapsulation means that the class should protect the integrity of the data it encapsulates. When data is required, it must often be supplied through a constructor. Conversely, a default constructor implies that no external data is required. That's a rather weak statement about the invariants of the class.
With all of this in mind, I think the example that you provided would be a risky approach unless the default SMTP sender is a Null Object (for example: doesn't actually send anything). Otherwise there is too much information hidden from the composition root, which just opens the gate to unpleasant surprises.
Just having a conversation with someone in the office about using a business logic class to build up some data in order to test another class.
Basically, he has class A which takes a complex type as a parameter and then generates a collection of a different complex type as a result. He's written tests around this class already. Now he's moved on to testing another class (class B) which takes the result of class A then performs some logic on it.
He's asked the question, "should I use class A to build up a scenario to test class B with".
At first I said yes as class A has tests around it. But then I figured well what if there's some bugs in class A that we haven't found yet... so I guess there must be a better way to address this scenario.
Does anyone have any thoughts on this? Is it OK to use existing logic to save time writing other tests?
Regards,
James
Stances on that might differ. Generally, if code is tested and you assume it works, you're free to use it. This is especially true when using already tested methods to help with testing others (within single class/unit). However, as this happens within single unit it's a bit different from your case.
Now, when dealing with 2 separate classes I'd say you should avoid such approach. Purely for the reason that those two classes might not be related in obvious way or their context/scope of usage might vastly differ. As in, somebody might change class A without even knowing class B exists. Class B tests suddenly break even though no changes were made to B code. This brings unnecessary confusion and is situation you usually don't want to find yourself in.
Instead, I suggest creating helper method within tested class B file. With stubs/fakes and tools like AutoFixture, you should be able to easily reproduce generation logic used by class A and have your own "copy" contained in class B tests.
In order to test class B, the result returned from the class A should be replicated somewhere in your test. If class A returns a list of persons, you will have an helper function in your test returning a fake List<Person> to use for your test.
Because you are only testing the class B, class A should not be used in your test.
NUnit provides a built in functionality in order to provide data for your tests, have a look to :
http://www.nunit.org/index.php?p=testCaseSource&r=2.5
Or you can simply create a DataFactory class with methods that returns the data (simple objects, collections etc..) you will consume in your tests.
This is in part a followup to this question.
I'm not sure the best way to ask this, so I'll try a short story to set the scene:
Once upon a time, there was a class ‘A’, which had a unit test class ‘ATests’ responsible for testing its behaviour through the public interface. They lived happily together for a while and then a change happened, and class ‘B’ came along, which as it turned out had a lot in common with class ‘A’, so a base class was introduced.
The public behaviour of the base class is already covered by the tests for class A. So, the question is what happens next?
• Does class B need to have tests for the common (base class behaviour)? It seems like the behaviour is a part of B, so it should be tested, but should these tests be shared with those for class A? For the base class? If so, what’s the best way to share?
• Does the new base class need unit tests, or is it ok for base classes to be tested through the tests of their children? Does it matter if the base class is abstract?
• Is it enough to ensure that classes A & B derive from the base class and ‘trust’ the unit tests for the base class to test the common behaviour (so the tests don’t need to be replicated in the child classes)? The tests for A & B only need to test they’re new/changed behaviour?
• Am I following totally the wrong approach having approximately one unit test class per real class?
I’ve taken different views at different times and the different approaches can have quite an impact on the ability to refactor the code, time taken to write tests etc. What approaches have people found works best?
Personally, given time, I tend to test all three (base and two derived). It shows that you're not inadvertently overriding the base methods and changing their behavior, and your inherited class still provides the expected semantics. If the behavior really doesn't change, then it could be as simple as a copy-paste job, but it provides more complete coverage.
Note the "given time" part, though. If time is an issue (and it always is), testing the base class or the inherited functionality would probably be lower priority. But testing is great inoculation against yourself, and makes you much more confident when refactoring later, so you're only shortchanging yourself, your customers, and/or your maintainers by not doing as complete coverage as you have time for.
However, pawning repetitive things like this off on dedicated testers or a QA team, if you have one, is perfectly acceptable. But buy them a beer sometimes :-) They make you look better!
You might look at code coverage tools; they can show you if you're actually testing all of the code. Personally if I have a test covering the base class behavior and I"m not overriding that, I won't test it again. The goal is to have a code change (potentially) break only one test.
Don't feel the need to stick to one unit test class per real class. One unit test class per unique setup fixture is one way, and then there's the whole BDD school...
Refactoring test code is just as important as refactoring production code. Both should be treated as first class citizens. So if you are extracting public methods to the base class then that should have its own set of tests. If your test cases are designed properly where each tests tests one thing each then the test refactoring should be easy.
If you are extracting protected functionality then probably its it a slightly grey area. If test methods are new to the class then I would expect them to be tested in the derived class simply because they are probably there for the derived class to function properly. Ideally this should be kept to a minimum as the base class functionality becomes less obvious.
The derived classes then will have the remaining public methods tested in their own set of tests.
Again if you change the production code and not the tests then you should incorporate a coverage tool so you feel confident enough your tests are covered enough.
This question already has answers here:
How do I test a class that has private methods, fields or inner classes?
(58 answers)
Closed 5 years ago.
In C++, I have often made a unit test class a friend of the class I am testing. I do this because I sometimes feel the need to write a unit test for a private method, or maybe I want access to some private member so I can more easily setup the state of the object so I can test it. To me this helps preserve encapsulation and abstraction because I am not modifying the public or protected interface of the class.
If I buy a third party library, I wouldn't want its public interface to be polluted with a bunch of public methods I don't need to know about simply because the vendor wanted to unit test!
Nor do I want have to worry about a bunch of protected members that I don't need to know about if I am inheriting from a class.
That is why I say it preserves abstraction and encapsulation.
At my new job they frown against using friend classes even for unit tests. They say because the class should not "know" anything about the tests and that you do not want tight coupling of the class and its test.
Can someone please explain these reasons to me more so that I may understand better? I just do not see why using a friend for unit tests is bad.
Ideally, you shouldn't need to unit test private methods at all. All a consumer of your class should care about is the public interface, so that's what you should test. If a private method has a bug, it should be caught by a unit test that invokes some public method on the class which eventually ends up calling the buggy private method. If a bug manages to slip by, this indicates that your test cases don't fully reflect the contract you wish your class to implement. The solution to this problem is almost certainly to test public methods with more scrutiny, not to have your test cases dig into the class's implementation details.
Again, this is the ideal case. In the real world, things may not always be so clear, and having a unit testing class be a friend of the class it tests might be acceptable, or even desirable. Still, it's probably not something you want to do all the time. If it seems to come up often enough, that might a sign that your classes are too large and/or performing too many tasks. If so, further subdividing them by refactoring complex sets of private methods into separate classes should help remove the need for unit tests to know about implementation details.
You should consider that there are different styles and methods to test: Black box testing only tests the public interface (treating the class as a black box). If you have an abstract base class you can even use the same tests against all your implementations.
If you use White box testing, you might even look at the details of the implementation. Not only about which private methods a class has, but what kind of conditional statements are included (i.e. if you want to increase your condition coverage because you know that the conditions were hard to code). In white box testing, you definitely have "high coupling" between classes/implementation and the tests which is necessary because you want to test the implementation and not the interface.
As bcat pointed out, it's often helpful to use composition and more but smaller classes instead of many private methods. This simplifies white box testing because you can more easily specify the test cases to get a good test coverage.
I feel that Bcat gave a very good answer, but I would like to expound on the exceptional case that he alludes to
In the real world, things may not always be so clear, and having a
unit testing class be a friend of the class it tests might be
acceptable, or even desirable.
I work in a company with a large legacy codebase, which has two problems both of which contribute to making a friend unit-test desirable.
We suffer from obscenely large functions and classes which require refactoring, but in order to refactor it is helpful to have tests.
Much of our code is dependent on database access, which for various reasons should not be brought into the unit tests.
In some cases Mocking is useful to alleviate the latter problem, but very often this just leads to uneccessarily complex design (class heirarchies where none would otherwise be needed), while one could very simply refactor the code in the following way:
class Foo{
public:
some_db_accessing_method(){
// some line(s) of code with db dependance.
// a bunch of code which is the real meat of the function
// maybe a little more db access.
}
}
Now we have the situation where the meat of the function needs refactoring, so we'd like a unit test. It shouldn't be exposed publicly. Now, there's a wonderful technique called mocking that could be used in this situation, but the fact is that in this case a mock is overkill. It would require me to increase the complexity of the design with an unecessary hierarchy.
A far more pragmatic approach would be to do something like this:
class Foo{
public:
some_db_accessing_method(){
// db code as before
unit_testable_meat(data_we_got_from_db);
// maybe more db code.
}
private:
unit_testable_meat(...);
}
The latter gives me all of the benefits I need from unit testing, including giving me that precious safety net to catch errors produced when I refactor the code in the meat. In order to unit test it, I have to friend a UnitTest class, but I would strongly argue that this is is far better than an otherwise useless code heirarchy just to allow me to use a Mock.
I think this should become an idiom, and I think it's a suitable, pragmatic solution to increase the ROI of unit testing.
Like bcat suggested, as much as possible, you need to find bugs using public interface itself. But if you want to do things like printing private variables and comparing with expected result etc(Helpful for developers to debug the issues easily), then you can make UnitTest class as friend to class to be tested. But you may need to add it under a macro like below.
class Myclass
{
#if defined(UNIT_TEST)
friend class UnitTest;
#endif
};
Enable flag UNIT_TEST only when Unit testing is required.
For other releases, you need to disable this flag.
I don't see anything wrong with using a friend unit testing class in many cases. Yes, decomposing a large class into smaller ones is sometimes a better way to go. I think people are a bit too hasty to dismiss using the friend keyword for something like this - it might not be ideal object oriented design, but I can sacrifice a little idealism for better test coverage if that's what I really need.
Typically you only test the public interface so that you are free to redesign and refactor the implementation. Adding test cases for private members defines a requirement and restriction on the implementation of your class.
Make the functions you want to test protected.
Now in your unit test file, create a derived class.
Create public wrapper functions that call your the class-under-test protected functions.
I am a fairly new software developer currently working adding unit tests to an existing C++ project that started years ago. Due to a non-technical reason, I'm not allowed to modify any existing code. The base class of all my modules has a bunch of methods for Setting/Getting data and communicating with other modules.
Since I just want to unit testing each individual module, I want to be able to use canned values for all my inter-module communication methods. I.e. for a method Ping() which checks if another module is active, I want to have it return true or false based on what kind of test I'm doing. I've been looking into Google Test and Google Mock, and it does support mocking non-virtual methods. However the approach described (https://google.github.io/googletest/gmock_cook_book.html#MockingNonVirtualMethods) requires me to "templatize" the original methods to take in either real or mock objects. I can't go and templatize my methods in the base class due to the requirement mentioned earlier, so I need some other way of mocking these virtual methods
Basically, the methods I want to mock are in some base class, the modules I want to unit test and create mocks of are derived classes of that base class. There are intermediate modules in between my base Module class and the modules that I want to test.
I would appreciate any advise!
Thanks,
JW
EDIT: A more concrete examples
My base class is lets say rootModule, the module I want to test is leafModule. There is an intermediate module which inherits from rootModule, leafModule inherits from this intermediate module.
In my leafModule, I want to test the doStuff() method, which calls the non virtual GetStatus(moduleName) defined in the rootModule class. I need to somehow make GetStatus() to return a chosen canned value. Mocking is new to me, so is using mock objects even the right approach?
There are some different ways of replacing non-virtual functions. One is to re-declare them and compile a new test executable for each different set of non-virtual functions you'd like to test. That's hardly scaleable.
A second option is to make them virtual for test. Most compilers allow you to define something on the command-line so compile your code with -DTEST_VIRTUAL=virtual or -DTEST_VIRTUAL to make them either virtual or normal depending on whether or not it's under test or not.
A third option which may be usable is to use a mocking framework that lets you mock non-virtual functions. I'm the author of HippoMocks (disclaimer with regard to neutrality and so on) and we've recently added the ability to mock plain C functions on X86 platforms. This can be extended to non-virtual member functions with a bit of work and would be what you're looking for. Keep in mind that, if your compiler can see both the use and the definition of a function at one time that it may inline it and that the mocking may fail. That holds in particular for functions that are defined in headers.
If regular C function mocking is sufficient for you, you can use it as it is now.
I would write a Perl/Ruby/Python script to read in the original source tree and write out a mocked source tree in a different directory. You don't have to fully parse C++ in order to replace a function definition.
One approach would be to specify different sources for testing. Say your production target uses rootModule.h and rootModule.cpp. Use different sources for your testing target. You can specify a different header by changing your include path, so that #include "rootModule.h" actually loads unittest/rootModule.h. Then mock rootModule to your heart's content.