I am investigating using mocking for unit tests I'm adding to existing code. For this I'm using HippoMocks. This involves another class calling some methods on my mock (that are all virtual). I want to avoid overspecifying all this but HippoMocks keep throwing NotImplementedException whenever the other class calls functions on my mock that I have not specified.
The below code exposes my issue.
void test()
{
class SimpleClassToMock
{
public:
virtual void memberFunction1() {}
virtual void memberFunction2() {}
};
MockRepository mocks;
// true or false here makes no difference.
mocks.autoExpect = true;
SimpleClassToMock* m = mocks.Mock<SimpleClassToMock>();
// I care about this function getting called.
mocks.ExpectCall(m, SimpleClassToMock::memberFunction1);
m->memberFunction1();
// HippoMocks fails on the next line by throwing NotImplementedException.
m->memberFunction2();
}
Is there any way to tell HippoMocks not to fail here? I only want to specify the expectations for things I care about for a particular test, not every single thing that is called.
PS: To those that have mocking experience, am I thinking about this all wrong? Is overspecifying the test in cases such as this not a problem/"what you want"?
To avoid overspecifying, you can use OnCall to allow them to be called 0-N times (optionally with argument checks, order checks and so on).
Related
If i need to test if a method within class under test has been called or not, can it be done without Mockito (or any mocking tool for that matter)?
Reason asking is that wherever i read about Mockito and similar tools, it says one should never mock CUT but its dependencies (that part is clear).
So, if thats the case then there are only 2 options left:
there is some other way of testing it without mocking
or
the fact the method was called should not be tested itself but some side effect or methods return value
For example (trivial and non-realworld), class MyClass can have 2 methods: A() and B(). A conditionay calls B based on some internal state.
After arranging state & acting by calling A() we want to assert that B() was called.
Either its not possible without mocking the whole CUT or 2 methods like this in a single class are always SRP violation smell and call for redesign where B() should actually be (mocked) dependency of MyClass CUT.
So, whats correct?
Usually I tend to not even use spies, instead I prefer to write my code in a way that for any class I write:
I test only non-private methods, since they're entry points into the class under test. So, in your example, if a() calls b(), maybe b() should be be private and, as a consequence, should not be tested. To generalize, a() is something that a class "can do" (a behavior), so I test the behavior, and not the method itself. If this behavior internally calls other things - well, its an internal matter of that class, if possible I don't make any assumptions on how does the class work internally, and always prefer "white-box" testing.
I only test "one" non-private method in a test.
All the methods should return something (best option) or at least call dependencies, or change internal state of the object under test. The list of dependencies is always clean-to-understand, I can't instantiate the object of CUT without supplying it a list of dependencies. For example, using constructor dependency injection is a good way of doing this. I mock only dependencies indeed, and never mock / spy CUT. Dependencies are never static but injected.
Now with these simple rules, the need to "test if a method within class under test has been called or not" basically can boil down to one of the following:
you're talking about private method. In this case - don't test it, test only public things.
The method is public - in this case you explicitly call it in unit test, so its irrelevant.
Now lets ask why do you want to test this if a method within CUT has been called or not?
If you want to make sure that it changed something. If this "something" is within the class - in other words, its internal state has changed, check in test that the change is indeed done in the state by calling another method that allows to query the state
If this "something" is a code that is managed by dependency, create a mock of this dependency and verify that it was called with the expected parameters.
Take a look at the Mockito Documentation (https://static.javadoc.io/org.mockito/mockito-core/3.0.0/org/mockito/Mockito.html#13)
When using a Spy you can 'replace' a method in the same class that is under test.
#ExtendWith(MockitoExtension.class)
public class Test {
class MyClass {
public void a() {
b();
}
public void b() {
}
}
#Test
public void test() {
MyClass testClass = new MyClass();
MyClass spy = Mockito.spy(testClass);
Mockito.doNothing().when(spy).b();
spy.a();
Mockito.verify(spy, Mockito.times(1)).b();
}
}
So whether that is something that should be done is a different question ;)
I think it highly depends on what method B() is actually doing and whether that is supposed be part of MyClass in the first place.
Either its not possible without mocking the whole CUT
In this case we do not mock the whole CUT only the method you do not want to be called.
Reason asking is that wherever i read about Mockito and similar tools, it says one should never mock CUT but its dependencies (that part is clear).
I believe this statement is not entirely accurate in correlation with spying.
The whole point of spying in my eyes is to use it on the class under test. Why would one want to spy on a dependecy that is not even supposed to be part of the test in the first place?
I am a long time lurker in the board, and is needless to say that you guys are the best and I am grateful for all the times you saved my job. That is my first time post here and I hope I don't mess it up.
I am writing a C++ Boost application for Linux (Virtualized Ubuntu 16.04 amd_64) and am using Turtle Mock for the mocking framework and Boost Test for the testing framework. When I try to test a class which uses dependency injection technique, I mock the classes that need to be given to the testing class so I can verify the sequence of their invocation. S far so good, but the problem comes here.
I am using MOCK_BASE_CLASS(MockAClass, AClass), to override the virtual methods of the real AClass, and use to new MockAClass to proceed with my tests. Let's say AClass has a virtual method int getTest(int), and MockAClass has MOCK_METHOD(getTest, 1, int(int)), after setting expectation and return value for the getTest method of MockAClass object, and invoking the method, the expectation which in most cases is MOCK_EXPECT(objMockAClass.getTest).at_least(1) is NEVER verified. I can control the return value, but the call is never verified as it happened. This only occurs if the function returns a value (for ex. if the function is void getTest(int) then the verification will pass).
I am attaching a simple PoC of my problem that will fail on my system.
class AClass
{
public:
virtual int getTest(int a) {return 0}
}
MOCK_BASE_CLASS (MockAClass, AClass)
{
MOCK_METHOD(getTest, 1, int(int));
}
BOOST_AUTО_TEST_CASE(SomeClassFunctionality)
{
MockAClass objMockAClass;
MOCK_EXPECT(objMockAClass.getTest).returns(1);
MOCK_EXPECT(objMockAClass.getTest).at_least(1);
objMockAClass.getTest(1);
}
MOCK_EXPECT(objMockAClass.getTest).returns(1);
MOCK_EXPECT(objMockAClass.getTest).at_least(1);
This is actually two expectations. The first one means 'everytime getTest gets called return 1' and the second 'getTest must be called at least one time'.
The problem is that the first one will always match, therefore the second will not have a chance to be triggered.
Тhe problem is solved if separate EXPECT statements are combined in one whole EXPECT statement.
MOCK_EXPECT(objMockAClass.getTest).at_least(1).returns(1); - This will make the example work as planned.
Regards,
I want to test if a method has been called in the test.
My problem is that when I want to create the expectations it is not working as I thought. The next line actually runs the method, not only create an expectation:
Expect.Call(() => mockedService.MethodThatIExpectToRun(params));
There is another way:
mockedService.Expect((s=> s.MethodThatIExpectToRun(params)));
But this also actually runs the method, not only creates an expectation to be fulfilled by the test.
And this line that asserts if the method was not called also actually calls the method, not only checks whether it was called.
mockedService.AssertWasCalled(s=> s.MethodThatIExpectToRun((params)));
additional info:
MethodThatIExpectToRun returns void
For prgmtc's comment:
IService mockedService = MockRepository.GeneratePartialMock<Service>(mockedRepository_1, ..., mockedRepository_n);
Usually when you have PartialMock that's calling the real method when setting a Stub or Expect, it means the virtual keyword is missing on the method.
Make sure Service.MethodThatIExpectToRun is virtual.
A more general (although perhaps less useful) remark: Partial mocks can point to a design smell of the code under test. If you only want to provide expectations for a part of a class, then perhaps that class has too many responsibilities and should be split into multiple classes? That way you can cleanly isolate responsibilities/collaborators and have no need for the partialmock construct. If you mock (not partial mock) an interface or virtual member of a concrete class the call won't go through to the real implementation.
For example:
Console.WriteLine("Real object: ");
new Foo().Bar();
Console.WriteLine("Mocked object: ");
var aFoo = MockRepository.GenerateMock<Foo>();
aFoo.Expect(f => f.Bar());
aFoo.Bar();
...
public class Foo
{
public virtual void Bar()
{
Console.WriteLine("REAL IMPLEMENTATION");
}
}
Outputs:
Real object:
REAL IMPLEMENTATION
Mocked object:
I often face the problem that mock objects need to be brought in a certain state before the "interesting" part of a test can start.
For example, let's say I want to test the following class:
struct ToTest
{
virtual void onEnable();
virtual void doAction();
};
Therefore, I create the following mock class:
struct Mock : ToTest
{
MOCK_METHOD0(onEnable, void());
MOCK_METHOD0(doAction, void());
};
The first test is that onEnable is called when the system that uses a ToTest object is enabled:
TEST(SomeTest, OnEnable)
{
Mock mock;
// register mock somehow
// interesting part of the test
EXPECT_CALL(mock, onEnable());
EnableSystem();
}
So far, so good. The second test is that doAction is called when the system performs an action and is enabled. Therefore, the system should be enabled before the interesting part of the test can start:
TEST(SomeTest, DoActionWhenEnabled)
{
Mock mock;
// register mock somehow
// initialize system
EnableSystem();
// interesting part of the test
EXPECT_CALL(mock, doAction());
DoSomeAction();
}
This works but gives an annoying warning about an uninteresting call to onEnable. There seem to be two common fixes of this problem:
Using NiceMock<Mock> to suppress all such warnings; and
Add an EXPECT_CALL(mock, onEnable()) statement.
I don't want to use the first method since there might be other uninteresting calls that really should not happen. I also don't like the second method since I already tested (in the first test) that onEnable is called when the system is enabled; hence, I don't want to repeat that expectation in all tests that work on enabled systems.
What I would like to be able to do is say that all mock calls up to a certain point should be completely ignored. In this example, I want expectations to be only checked starting from the "interesting part of the test" comment.
Is there a way to accomplish this using Google Mock?
The annoying thing is that the necessary functions are there: gmock/gmock-spec-builders.h defines Mock::AllowUninterestingCalls and others to control the generation of warnings for a specific mock object. Using these functions, it should be possible to temporarily disable warnings about uninteresting calls.
That catch is, however, that these functions are private. The good thing is that class Mock has some template friends (e.g., NiceMock) that can be abused. So I created the following workaround:
namespace testing
{
// HACK: NiceMock<> is a friend of Mock so we specialize it here to a type that
// is never used to be able to temporarily make a mock nice. If this feature
// would just be supported, we wouldn't need this hack...
template<>
struct NiceMock<void>
{
static void allow(const void* mock)
{
Mock::AllowUninterestingCalls(mock);
}
static void warn(const void* mock)
{
Mock::WarnUninterestingCalls(mock);
}
static void fail(const void* mock)
{
Mock::FailUninterestingCalls(mock);
}
};
typedef NiceMock<void> UninterestingCalls;
}
This lets me access the private functions through the UninterestingCalls typedef.
The flexibility you're looking for is not possible in gmock, by design. From the gmock Cookbook (emphasis mine):
[...] you should be very cautious about when to use naggy or strict mocks, as they tend to make tests more brittle and harder to maintain. When you refactor your code without changing its externally visible behavior, ideally you should't need to update any tests. If your code interacts with a naggy mock, however, you may start to get spammed with warnings as the result of your change. Worse, if your code interacts with a strict mock, your tests may start to fail and you'll be forced to fix them. Our general recommendation is to use nice mocks (not yet the default) most of the time, use naggy mocks (the current default) when developing or debugging tests, and use strict mocks only as the last resort.
Unfortunately, this is an issue that we, and many other developers, have encountered. In his book, Modern C++ Programming with Test-Driven Development, Jeff Langr writes (Chapter 5, on Test Doubles):
What about the test design? We split one test into two when we changed from a hand-rolled mock solution to one using Google Mock. If we expressed everything in a single test, that one test could set up the expectations to cover all three significant events. That’s an easy fix, but we’d end up with a cluttered test.
[...]
By using NiceMock, we take on a small risk. If the code later somehow changes to invoke another method on the [...] interface, our tests aren’t going to know about it. You should use NiceMock when you need it, not habitually. Seek to fix your design if you seem to require it often.
You might be better off using a different mock class for your second test.
class MockOnAction : public ToTest {
// This is a non-mocked function that does nothing
virtual void onEnable() {}
// Mocked function
MOCK_METHOD0(doAction, void());
}
In order for this test to work, you can have onEnable do nothing (as shown above). Or it can do something special like calling the base class or doing some other logic.
virtual void onEnable() {
// You could call the base class version of this function
ToTest::onEnable();
// or hardcode some other logic
// isEnabled = true;
}
What is the best way of testing a function that throws on failure? Or testing a function that is fairly immune to failure?
For instance; I have a I/O Completion Port class that throws in the constructor if it can't initialise the port correctly. This uses the Win32 function of CreateIoCompletionPort in the initialiser list. If the handle isn't set correctly - a non-null value - then the constructor will throw an exception. I have never seen this function fail.
I am pretty certain that this (and other functions like it in my code) if they fail will behave correctly, the code is 50 lines long including white-space, so my questions are
a) is it worth testing that it will throw
b) and if it is worth testing, how to?
c) should simple wrapper classes as these be unit-tested?
For b) I thought about overriding CreateIoCompletionPort and passing the values through. In the unit test override it and cause it to return 0 when a certain value is passed in. However since this is used in the constructor then this needs to be static. Does this seem valid or not?
If you are doing this in .NET, there is an ExpectedException attribute that you can add to your test:
[Test, ExpectedException(typeof(SpecificException), "Exception's specific message")]
public void TestWhichHasException()
{
CallMethodThatThrowsSpecificException();
}
Test will pass if the exception of that type and with the specified message is thrown. The attribute has other overloads including having InnerExceptions, etc.
It is definitely worthwhile to test failure conditions, both that your class properly throws an exception when you want it to and that exceptions are handled properly in the class.
This can easily be done if you are acting on an object passed in to the constructor... just pass in a mock. If not, I tend to prefer to have the functionality moved to a protected method, and override the protected method to evoke my failure case. I will use Java as an example, but it should be easy enough to port the ideas to a C# case:
public class MyClass {
public MyClass() throws MyClassException {
// Whatever, including a call to invokeCreateIoCompletionPort
}
protected int invokeCreateIoCompletionPort(String str, int i) {
return StaticClass.createIoCompletionPort(str, i);
}
}
public class MyTest {
public void myTest() {
try {
new MyClass();
fail("MyClassException was not thrown!");
} catch (MyClassException e) {
}
}
private static class MyClassWrapper extends MyClass {
#Override
protected int invokeCreateIoCompletionPort(String str, int i) {
throw new ExpectedException();
}
}
}
As you can see, it is pretty easy to test whether an exception is being thrown by the constructor or method you are testing, and it is also pretty easy to inject an exception from an external class that can throw an exception. Sorry I'm not using your actual method, I just used the name to illustrate how it sounded like you are using it, and how I would test the cases it sounded you wanted to test.
Basically, any API details you expose can usually be tested, and if you want to KNOW that exceptional cases work as they should, you probably will want to test it.
You should consider writing your code in such a way that you can mock your I/O completion port. Make an interface/abstract class that exposes the methods you need on the I/O object, and write and test implementation that does things like it's supposed to (and an option to simulate failure perhaps).
AFAIK it's a common practice to mock external resources when unit testing, to minimize dependencies.
Sound like C++ to me. You need a seam to mock out the Win32 functions. E.g. in your class you would create a protected method CreateIoCompletionPort() which calls ::CreateIoCompletionPort() and for your test you create a class that derives from you I/O Completion Port class and overrides CreateIoCompletionPort() to do nothing but return NULL. Your production class is still behaving like it was designed but you are now able to simulate a failure in the CreateIoCompletionPort() function.
This technique is from Michael Feathers book "Working effectively with legacy code".