In all the documentation about gmock I always find the mock object to be instantiated inside a test, like that:
TEST(Bim, Bam)
{
MyMockClass myMockObj;
EXPECT_CALL(MyMockObj, foo(_));
...
}
So, the object is created and destroyed per test. I believe it's also perfectly fine to create and destroy the object per test fixture. But I'm wondering if it's also ok to have a file-global instance of the mock object, like that:
MyMockClass myMockObj;
TEST(Bim, Bam)
{
EXPECT_CALL(MyMockObj, foo(_))
...
}
I tried it and I have absolutely no problems so far, it all seems to work fine. But maybe I should be aware of anything? Just because I stumbled about this question, where the only answer states:
... the problem is that you're instantiating a global instance of FooMock. Googlemock/googletest expect the mock to be defined either within the body of the test, or within a test fixture class.
But I could not find anything in the documentation or anywhere else that confirms this (did I overlook it?).
Thanks, Georg
PS: The reason why I need to use a global mock instance would be the topic of another discussion (see this posting of mine).
You can, but it is not a good idea.
Doing such a thing is violate the isolation principle of UT.
This violation may cause an unexpected failure/pass in your tests.
Gtest uses the destructor of the fake objects to verify that the expectation occurred, this is the reason behind the expectation that each fake object will create and release in the body of the test, or within a test fixture class.
If you make the fake object global then it won't release at the end of each UT, then the verification won't execute and the test will pass even when it should fail. more over some of your UTs may fass/fail when you execute all your tests together; in one test you expect the method x won't call and in the other you expect that the method will call; in one UT you expect the method x will call 3 times, but the method was call twice in the test + one in other test(the test should fail but it won't...)
So the bottom line you should never use a global mock unless this global mock is being in use only to prevent null pointer(you didn't set a behaviour..)
Just stumbled across this question while chasing a bug related to my mock objects. In my case the problem was that the mock object's constructor was being called before InitGoogleMock, and that seemed to send things off into the weeds.
Note: I'm using Google Mock with CppUnitTestFramework.
Fail:
MockObject mock;
TEST_MODULE_INITIALIZE(ModuleInitialize)
{
InitGoogleMock(argc, argv);
}
Win:
MockObject *mock = nullptr;
TEST_MODULE_INITIALIZE(ModuleInitialize)
{
InitGoogleMock(argc, argv);
mock = new MockObject;
}
TEST_MODULE_CLEANUP(ModuleCleanup)
{
delete mock;
}
Not saying it's best practice or anything, but if you need global mock objects I'd say pay attention to when your constructors are being called.
In addition to the accepted answer, if you're using GTest, global variables will also be tagged as leakage when they're not destroyed after test cases are executed.
The idea behind leakage is in this reference: https://google.github.io/googletest/gmock_cook_book.html#forcing-a-verification
If you don't want manual verification, the closest solution would be to have the mock object as a member of your fixture class. And if for some reason you would need to dynamically allocate mock, you can have a pointer and create/destroy the instance on SetUp and TearDown (the same concept as #Chris Olsen's answer). Or if you're in C++11, you can use shared_ptr:
class Fixture : public ::testing::Test
{
std::shared_ptr<ObjT> mPtr;
...
void SetUp()
{
mPtr = std::make_shared<ObjT>();
}
...
}
Related
If i need to test if a method within class under test has been called or not, can it be done without Mockito (or any mocking tool for that matter)?
Reason asking is that wherever i read about Mockito and similar tools, it says one should never mock CUT but its dependencies (that part is clear).
So, if thats the case then there are only 2 options left:
there is some other way of testing it without mocking
or
the fact the method was called should not be tested itself but some side effect or methods return value
For example (trivial and non-realworld), class MyClass can have 2 methods: A() and B(). A conditionay calls B based on some internal state.
After arranging state & acting by calling A() we want to assert that B() was called.
Either its not possible without mocking the whole CUT or 2 methods like this in a single class are always SRP violation smell and call for redesign where B() should actually be (mocked) dependency of MyClass CUT.
So, whats correct?
Usually I tend to not even use spies, instead I prefer to write my code in a way that for any class I write:
I test only non-private methods, since they're entry points into the class under test. So, in your example, if a() calls b(), maybe b() should be be private and, as a consequence, should not be tested. To generalize, a() is something that a class "can do" (a behavior), so I test the behavior, and not the method itself. If this behavior internally calls other things - well, its an internal matter of that class, if possible I don't make any assumptions on how does the class work internally, and always prefer "white-box" testing.
I only test "one" non-private method in a test.
All the methods should return something (best option) or at least call dependencies, or change internal state of the object under test. The list of dependencies is always clean-to-understand, I can't instantiate the object of CUT without supplying it a list of dependencies. For example, using constructor dependency injection is a good way of doing this. I mock only dependencies indeed, and never mock / spy CUT. Dependencies are never static but injected.
Now with these simple rules, the need to "test if a method within class under test has been called or not" basically can boil down to one of the following:
you're talking about private method. In this case - don't test it, test only public things.
The method is public - in this case you explicitly call it in unit test, so its irrelevant.
Now lets ask why do you want to test this if a method within CUT has been called or not?
If you want to make sure that it changed something. If this "something" is within the class - in other words, its internal state has changed, check in test that the change is indeed done in the state by calling another method that allows to query the state
If this "something" is a code that is managed by dependency, create a mock of this dependency and verify that it was called with the expected parameters.
Take a look at the Mockito Documentation (https://static.javadoc.io/org.mockito/mockito-core/3.0.0/org/mockito/Mockito.html#13)
When using a Spy you can 'replace' a method in the same class that is under test.
#ExtendWith(MockitoExtension.class)
public class Test {
class MyClass {
public void a() {
b();
}
public void b() {
}
}
#Test
public void test() {
MyClass testClass = new MyClass();
MyClass spy = Mockito.spy(testClass);
Mockito.doNothing().when(spy).b();
spy.a();
Mockito.verify(spy, Mockito.times(1)).b();
}
}
So whether that is something that should be done is a different question ;)
I think it highly depends on what method B() is actually doing and whether that is supposed be part of MyClass in the first place.
Either its not possible without mocking the whole CUT
In this case we do not mock the whole CUT only the method you do not want to be called.
Reason asking is that wherever i read about Mockito and similar tools, it says one should never mock CUT but its dependencies (that part is clear).
I believe this statement is not entirely accurate in correlation with spying.
The whole point of spying in my eyes is to use it on the class under test. Why would one want to spy on a dependecy that is not even supposed to be part of the test in the first place?
I have below code, just putting one scenario here.
class A{
public JSONObject m1(type1,type2,type3){
callmethod2(type3);
}
public Map callmethod2(type3){
//some jobs
return myMap;
}
}
#Test
assertequals(JSONObjectTest,m1(type1,type2,type3))
In my test, I am creating a mock with dummy data and passing them to actual function type1,type2,type3 Now there is callmethod2(type3) as a dependency so i do not want to execute that method. so in my test case i written
when(mockA.callmethod2(any(type3.class))).thenReturn(mockMap);
But i can see my callmethod2 is getting execute, how can i solve this and set some expected result for that method in my test case, so it will not execute.
Your code scenario at the time I am writing this is a little incomplete, but I will try to read between the lines. I assume you are somewhere creating a mock instance of class A. I am also assuming you are using that mocked instance to invoke the instance method "m1".
If either of these things are not true, then you will have problems. If you do not create a mock instance, then the "when(...)" clause will not function properly, and in fact while you are invoking the "when" clause you will also actually be invoking the real (i.e., unmocked) instance of A.m1(). The same will be true if you properly mock A but then don't use the mocked instance in the "when" clause, or if you don't use the mocked version in your execute/assert statement(s).
But the bigger problem is that if you actually ARE properly creating and using a mock of A, then even the default mocked functionality of method "m1" becomes empty -- do nothing. It would never get executed. The only way to do what you seem to be proposing is to use what Mockito calls "partial mocking", using a Spy. You instantiate a real instance of A, then wrap it in a Spy to make a "Spied"-instance of A. Now the Spied instance of A inherits all of the default implementations of methods, and you can mock out specific methods if you like. You then must use the spied instance in both yur "whe(...)" clauses as well as your execute/assert statements.
But even that endeavor is questionable. You should try to test classes as a whole rather than mocking out partial behavior. In general (practically a rule),you should either mock nor spy the class under test. If you see a need to do so then there is probably a coherency problem with your class, and it is pointing to design issues or a misunderstanding of good testing practices.
In mocking frameworks, you can mock an object or spy on it. What's the difference between the two and when would/should I use one over the other?
Looking at Mockito, for example, I see similar things being done using spies and mocks, but I am unsure as to the distinction between the two.
Mock object replace mocked class entirely, returning recorded or default values. You can create mock out of "thin air". This is what is mostly used during unit testing.
When spying, you take an existing object and "replace" only some methods. This is useful when you have a huge class and only want to mock certain methods (partial mocking). Let me quote Mockito documentation:
You can create spies of real objects. When you use the spy then the real methods are called (unless a method was stubbed).
Real spies should be used carefully and occasionally, for example when dealing with legacy code.
When in doubt, use mocks.
I'll try to explain using an example here:
// Difference between mocking, stubbing and spying
#Test
public void differenceBetweenMockingSpyingAndStubbing() {
List list = new ArrayList();
list.add("abc");
assertEquals(1, list.size());
List mockedList = spy(list);
when(mockedList.size()).thenReturn(10);
assertEquals(10, mockedList.size());
}
Here, we had initial real object list, in which we added one element and expected size to be one.
We spy real object meaning that we can instruct which method to be stubbed. So we declared that we stubbed method - size() on spy object which will return 10, no matter what is actual size.
In a nutshell, you will spy real object and stub some of the methods.
Mockito warns that partial mocking isn't a good practice and you should revise your Object Oriented architecture. Spy (or partial mocking) is recommended to test legacy code.
Based on Mocks Aren't Stubs by Martin Fowler:
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test.
Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent.
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
In Mockito if you assign any object to instance variable of Mock Object then does not affect on Mock Object.
But in case of Spy, if you assign any object to instance variable of Spy Object then does affect on Spy Object because of Spy act like real-time object modification.
For a reference example are
#RunWith(MockitoJUnitRunner.class)
public class MockSpyExampleTest {
#Mock
private List<String> mockList;
#Spy
private List<String> spyList = new ArrayList();
#Test
public void testMockList() {
//by default, calling the methods of mock object will do nothing
mockList.add("test");
assertNull(mockList.get(0));
}
#Test
public void testSpyList() {
//spy object will call the real method when not stub
spyList.add("test");
assertEquals("test", spyList.get(0));
}
}
Reference: http://javapointers.com/tutorial/difference-between-spy-and-mock-in-mockito/
When using mock objects, the default behavior of the method when not stub is do nothing. Simple means, if its a void method, then it will do nothing when you call the method or if its a method with a return then it may return null, empty or the default value.
While in spy objects, of course, since it is a real method, when you are not stubbing the method, then it will call the real method behavior. If you want to change and mock the method, then you need to stub it.
Spies have two definitions. One, is where the real method is called, another where, no functionality is called and only null or null equivalent values are returned, but methods were called, and they're state was recorded, commonly like, method x was called y times.
If we want to avoid calling external services and just want to test the logic inside the method then use mock.
If we want to call the external services and use the real dependency i.e to run the program as it is and just stub specific methods then use spy
Example
I have a repository class (DAL):
public class MyRepository : IMyRepository
{
public void Delete(int itemId)
{
// creates a concrete EF context class
// deletes the object by calling context.DeleteObject()
}
// other methods
}
I also have a service class (BLL):
public class MyService
{
private IMyRepository localRepository;
public MyService(IMyRepository instance)
{
this.localRepository = instance;
}
public void Delete(int itemId)
{
instance.Delete(itemId);
}
// other methods
}
Creating a unit test for MyRepository would take much more time than implementing it, because I would have to mock Entity Framework context.
But creating a unit test for MyService seems nonsense, because it only calls into Repository. All I could check is to verify if it did actually call repository Delete method.
Question
How would you suggest to unit test these pair of Delete methods. Both? One? None? And what would you test?
Yes, I would definitely write a unit test for the Service Layer. The reason for this is because, you're not just testing that your implementation works now, but you're also testing that it will continue to work in the future.
This is a vital concept to understand. If someone comes along later on and changes your ServiceLayer, and there's no unit test, how can you verify that the functionality continues to work?
I would also write tests for your DAL, but I would put those in a separate assembly called DataTests or something. The purpose here is to isolate your concerns across assemblies. Unit Tests shouldn't be concerned with your DAL, really.
Yes, both.
IMyRepository mock = ...;
// create Delete(int) expectation
MyService service = new MyService(mock);
service.Delete(100);
// Verify expectations
Your Delete method right now might only call the Delete method on the repository, but that doesn't mean it always will. You want to have unit tests for this partly to verify it behaves correctly and partly as way of defining your specifications of how the repository is to work.
You also aught to have a test that verifies that the constructor will throw an exception if the repository is null. You might also have other validation to do here in this method such as non-negative ID's, or non-zero id. Maybe that doesn't happen here, make it part of the specifications by creating tests that verify the expected behaviors.
They seem trivial but I can all but guarantee it will change one day and your expectation and specifications may not be verified.
Create the test for the Service. Currently all it does is to call into the Repository Delete method; however, you shouldn't care about that. What if later something happens and the functionality becomes much more complicated? Don't you want to have unit test code that will assure you that the functionality is still working as expected?
If you're exposing your Delete through your Service, you're expecting it to have an effect. Write a Unit Test to test that effect. Depending on your particular needs, I'd say you might not need to have a test on the Repository Delete, particularly if that functionality is getting exercised as part of your Service Delete functionality, but it really all depends on what level of coverage you're trying for.
Also, if you had created this code with TDD, you would have had a test. It actually matters whether people can call Delete through your service, so you actually have to test it.
In my opinion you need to test both. Maybe you can do the creation EF context class in a seperate factory that can be tested more easy and mock the context class for the MyRepository tests. That will be more easy and using a factory for creating a context calls seems to be quiet useful for me.
I'm using TypeMock Isolater to mock up some objects for some unit tests - attempting to use the AAA api (so the Isolate calls).
I have a straightforward singleton class where you call a static GetInstance(), which then returns an instance of the class. I thought it would be a simple matter of mocking that up, but I'm running into a very frustrating problem ! I can't seem to make GetInstance() return my mocked object correctly with my expected calls set.
I've tried:
using MST projects (using the Accessor classes) to assign a mocked object directly to the instance variable (faking the object using Memers.MustSpecifyReturnValues, and Isolate.WhenCalled using WithExactArguments to set expectations), but for some reason the mocked object always returns null (and no exceptions).
Mocking Singleton.GetInstance() to return the mocked object. This returns a mocked object which needs WhenCalled set, but now the Isolate.WhenCalled calls I make seem to do nothing on the fake object - so all calls throw an unexpected call exception.
I've also tried mocking the actual method call (eg Singleton.GetInstance().Test()), which will work for the call to that method, but all other calls to other methods on the singleton return null rather then throw an exception as I want it to (because this seems to automatically mock up all the objects without Members.MustSpecifyReturnValues).
All I want is to mock a singleton, and any calls I don't explicitly tell it to expect to throw an exception on. I thought it would be simple, but apparently not ! Sad
Has anyone any idea what I'm doing wrong?
Thanks
James
I think the simple solution will be to create a fake instance of the singleton class and use SwapNextInstace before the actual class constructor is called:
[TestMethod]
public void SetBhaciorOnSingleton()
{
var fake = Isolate.Fake.Instance<SingletonClass>();
Isolate.WhenCalled(() => fake.SomeFunction()).WillReturn(10);
// Set additional behavior on singleton class
Isolate.Swap.NextInstance<SingletonClass>().With(fake);
// This is where the class constructor is being called
var result = SingletonClass.GetInstace().SomeFunction();
Assert.AreEqual(10, result );
}
This solution should work with most scenarios unless the singleton class is created before the test.
If you need to set behavior after the class was created just use WhenCalled:
[TestMethod]
public void SetBhaciorOnSingleton()
{
var fake = Isolate.Fake.Instance<SingletonClass>();
Isolate.WhenCalled(() => fake.SomeFunction()).WillReturn(10);
Isolate.WhenCalled(() => SingletonClass.GetInstace()).WillReturn(fake);
var result = SingletonClass.GetInstace().SomeFunction();
Assert.AreEqual(10, result );
}
Disclaimer I work at Typemock.
You don't need to mock Singleton.GetInstance<>(). Using Isolate.Fake.AllInstances<>() instead of Isolate.Fake.Instance<>() you can mock the singleton. Then by setting the behavior on fake singleton behavior applied to all instances.
Take a look on the example:
public class Singleton
{
private Singleton() { }
static readonly Singleton instance = new Singleton();
public static Singleton Instance { get { return instance; } }
public int ReturnZero()
{
return 0;
}
}
[TestMethod]
public void FakeSingleton()
{
// Here we are setting the same behavior on all instances.
// The behavior we set on fake will apply to past instance as well
var fakeSingleton = Isolate.Fake.AllInstances<Singleton>();
Isolate.WhenCalled(() => fakeSingleton.ReturnZero()).WillReturn(10);
// Assert that the behavior works.
Assert.AreEqual(10, Singleton.Instance.ReturnZero());
}
Thanks.
I didn't try NextInstance before because it doesn't work on interfaces which I didn't really want to change.
But, I've tried it and it does work - although I was assuming the order of setting WhenCalled(s) doesn't really matter, but it definately does. If I do the WhenCalled after the Swap for instance, it doesn't work. It needs to go before the Swap. (Doesn't really make sense to me to be honest - it should be the same object).
However, the last example (one of the ways I had tried), doesn't work for me. I fake, set expecation on fake, and then set expectation on Singleton to return faked instance - but now it returns the concrete instance !
Could it have something to do with the way the constructors are called? I remember seeing something about that...
Alternatively I could use the Swap, but, I wanted to be able to setup all this stuff in a TestSetup, and make minor modifications to the expectations in the actual test, but that doesn't look possible. ?
The best solution is to not use singletons (or any other static mutable data). Just create one instance and use dependency injection to pass it to all objects who need it.
http://butunclebob.com/ArticleS.UncleBob.SingletonVsJustCreateOne
http://www.youtube.com/watch?v=-FRm3VPhseI