I have a C++ class that generates unique IDs in the following fashion.
class Foo
{
static int seed;
public:
const int Uid;
Foo() : Uid(seed++) {}
}
int Foo::seed = 0;
Now I'm using Google Test to test this Id generator using:
Foo foo;
EXPECT_EQ(0, foo.Uid);
Foo foo2;
EXPECT_EQ(1, foo2.Uid);
This test passes when I debug it but fails when I actually run it, giving me IDs of 2 and 3 instead. Can someone help me figure out why? Is Google test running two of these tests back-to-back or something?
One of characteristics of good tests is repeatability with not depending on tests order execution.
You have a singleton, and you use it in a bad way. If we assume there are no memory problems, then what most likely happens, is that the object of type Foo is somewhere created, and that your tests are getting executed in different order for debug and normal runs. That would explain different results.
How to fix? The simplest hack would be to add a method to reset the counter, and call it in setUp(). To fix it properly, you need to think how to remove that singleton.
BЈовић's answer will get you there. It's likely that somewhere else in your test code another instance of Foo is created that increases the value of the static member.
This may be a less hacky solution and possibly useful if adding a public reset method is not applicable (i.e. you don't want to add test-specific code to your API):
Mock classes. Add a class that dervies from Foo. Change the access level of your static variable from private to protected and add the new reset method to the protected class. You can then refactor your test code to run off of the derived class.
Make FooTester a friend of Foo. Via class friend ship you're allowing your test code more manipulation over the class under test.
I recommend going with the mock route. Keeps the original class clean of any test specific hacks and allows you to be more aware of any exposures in its interface.
Related
If i need to test if a method within class under test has been called or not, can it be done without Mockito (or any mocking tool for that matter)?
Reason asking is that wherever i read about Mockito and similar tools, it says one should never mock CUT but its dependencies (that part is clear).
So, if thats the case then there are only 2 options left:
there is some other way of testing it without mocking
or
the fact the method was called should not be tested itself but some side effect or methods return value
For example (trivial and non-realworld), class MyClass can have 2 methods: A() and B(). A conditionay calls B based on some internal state.
After arranging state & acting by calling A() we want to assert that B() was called.
Either its not possible without mocking the whole CUT or 2 methods like this in a single class are always SRP violation smell and call for redesign where B() should actually be (mocked) dependency of MyClass CUT.
So, whats correct?
Usually I tend to not even use spies, instead I prefer to write my code in a way that for any class I write:
I test only non-private methods, since they're entry points into the class under test. So, in your example, if a() calls b(), maybe b() should be be private and, as a consequence, should not be tested. To generalize, a() is something that a class "can do" (a behavior), so I test the behavior, and not the method itself. If this behavior internally calls other things - well, its an internal matter of that class, if possible I don't make any assumptions on how does the class work internally, and always prefer "white-box" testing.
I only test "one" non-private method in a test.
All the methods should return something (best option) or at least call dependencies, or change internal state of the object under test. The list of dependencies is always clean-to-understand, I can't instantiate the object of CUT without supplying it a list of dependencies. For example, using constructor dependency injection is a good way of doing this. I mock only dependencies indeed, and never mock / spy CUT. Dependencies are never static but injected.
Now with these simple rules, the need to "test if a method within class under test has been called or not" basically can boil down to one of the following:
you're talking about private method. In this case - don't test it, test only public things.
The method is public - in this case you explicitly call it in unit test, so its irrelevant.
Now lets ask why do you want to test this if a method within CUT has been called or not?
If you want to make sure that it changed something. If this "something" is within the class - in other words, its internal state has changed, check in test that the change is indeed done in the state by calling another method that allows to query the state
If this "something" is a code that is managed by dependency, create a mock of this dependency and verify that it was called with the expected parameters.
Take a look at the Mockito Documentation (https://static.javadoc.io/org.mockito/mockito-core/3.0.0/org/mockito/Mockito.html#13)
When using a Spy you can 'replace' a method in the same class that is under test.
#ExtendWith(MockitoExtension.class)
public class Test {
class MyClass {
public void a() {
b();
}
public void b() {
}
}
#Test
public void test() {
MyClass testClass = new MyClass();
MyClass spy = Mockito.spy(testClass);
Mockito.doNothing().when(spy).b();
spy.a();
Mockito.verify(spy, Mockito.times(1)).b();
}
}
So whether that is something that should be done is a different question ;)
I think it highly depends on what method B() is actually doing and whether that is supposed be part of MyClass in the first place.
Either its not possible without mocking the whole CUT
In this case we do not mock the whole CUT only the method you do not want to be called.
Reason asking is that wherever i read about Mockito and similar tools, it says one should never mock CUT but its dependencies (that part is clear).
I believe this statement is not entirely accurate in correlation with spying.
The whole point of spying in my eyes is to use it on the class under test. Why would one want to spy on a dependecy that is not even supposed to be part of the test in the first place?
I want to use Google test to test my class.
Lets assume I have a state machine implementation and the current state is private
so I have a method SetNextState that looks like that:
void setNextState
{
switch(m_currentState) //m_currentState is a private member
{
case INIT_STATE:
{
if some conditions occurred m_currentState=GO_STATE
}
......
}
}
so I have several cases and each define the behavior to move from certain state to another.
My question:
How do I perform tests on that method assuming the state is relevant only to this class so there is no output
How do I set its value to be, for example "GO_STATE" to test the GO_STATE case
and how do i check the m_currentState at the end of the test
Im trying to avoid putting friends etc. in my UUT code since I want it to be as original as possible
You don't. You do the same thing that your actual program will do, which is provide an input, then examine the result; you say there's no output, but there must be some effect, otherwise the class is pointless!
Failing that, you could make the test a "friend" of the class so that it can inspect its internals, or add an immutable getter for the current state (and who really cares if your class's users get to see that?) but neither option is really in the spirit of the thing.
In my experience, you'll occasionally realise that you're not really unit testing any more but instead functional testing, and Google Test may not be the right tool for that job. If your class is as big as it sounds, that could be the case here. Conversely, you could help yourself by splitting the class into smaller chunks, then unit testing those. Depends what you're going for, really.
Lightness Races in Orbit is correct. However, if sometimes you feel like it's useful to test the private member functions of your class, it often means that your class could be split in multiple smaller pieces.
If you don't think those smaller components are useful to the clients of your library, you can simply hide them in a detail:: namespace and then create unit tests as usual. This will allow you to test the internal behavior of your classes without polluting your public API.
After much considerations I decided to wrap my UUT with a helper which provides set and get to the relevant private members.and use it in the test procedure before calling the tested API
Original code
===============
class UUT //That's the actual class I want to test
{
protected:
int m_protectedMember;
public:
void methodToTest()
{
//Do something with m_protectedMember use its value as input
//and set it as output
}
};
In the tester
==============
class UUTHelper: public UUT
{
public:
int getProtectedMember() { return m_protectedMember; }
void setProtectedMember(int value) { m_protectedMember = value; }
};
The pros:
My test code is very simple and I easily create complicated scenarios .
I test the real code without any "friends" or any other manipulations.
The cons:
As written in the discussion, not the best "good practice", touching private members
Thank you all :)
I'm taking my first steps with unit testing and have a problem with encapsulation. My class has some private member variables that shouldn't be visible to the client, but in order for me to put object in a state I want to test it under, I need to set those private variables.
Say I have a code like that:
Class Foo {
public:
int action() ;
private:
int state ;
} ;
int Foo::action()
{
if(this->state == 1)
return 1 ;
else
return 0 ;
}
So now I want to test Foo::action(), but I need to be able to set Foo::state to be able to check function under different scenarios. One solution is the evil "define private public" in tests code. But is there something more elegant? I would like to stress that Foo::state is a variable that shouldn't be accessed by client, so I don't want to declare any public setter.
Edit:
I now think that extending the class I want to test in test code and including setters in that derived class would work, providing I changed private variables to protected. But that's a 'one generation only' solution and still feels like a hack rather than a proper approach.
Edit 2:
After reading answers and comments I was given (thanks to Lieven and ap. in particular) I believe the actual class I'm trying to test now (not the simple example I provided) simply does too much and the answer to my problem is moving some of its logic into another class that will be used by the big guy.
There are only two posibilities (refactoring asside)
Use the public interface to set the state.
The state is redundant if you can't set it to through the public interface.
Option 2 is self explanatory and most likely not applicable to your case so you are left with setting the state through the public interface of your class.
As you have already mentioned, this is possible but it requires a lot of code to get to the right state. That in itself could be an indication that your class is currently doing to much and it's time to refactor parts of your class into smaller, testable classes.
From Do not test private methods
If you find the need to test a private method, then you’re doing
something else wrong. There is an “upstream” problem, so to speak. You
have arrived at this problem due to some other mistake previous in
this process. Try to isolate what that is, exactly, and remove that
rather than bending your tests to go down a painful road of
brittleness in testing.
and Unit testing private members
I'd recommend not unit testing private methods. Since they're
private, they may be changed in any conceivable (or inconceivable?)
manner between each release. If a given private method is so critical
to the operation of the class that you feel it deserves test cases,
then it's probably time to refactor that out into a protected or
public method
A common quote on this is
You should never touch your privates
If your test class is called MyTestClass, then add MyTestClass as friend in class Foo to be able to access its private member variables.
Class Foo {
public:
int action();
private:
int state;
friend class MyTestClass;
};
You should be having some publicly (or "protectedly") accessible mechanism to change the value of the private variable state. For simplicity, let's say it is a method Foo::setState(int inState). Use that in the unit test to change the state, and thus test the Foo::action() method. This ensures that any future implementation changes would not affect the unit test (unless the "API" Foo::setState() changes - in which case, of course, you have to change the unit test).
If you do not have such a mechanism to change state, it means the end user or calling code cannot change it either, and hence, you wouldn't need to test it (and maybe that makes state redundant, but I don't know about that).
If the private variable changes "indirectly" through other code, you will have to execute the same code in the unit test. You can always trace back any method visible externally to inputs fed to the code externally. Basically the point is that in the unit test, you would have to feed the same inputs to the code as in the "real" scenario, and then test if it responds as it should.
As discussed in the comments below, if the "path" from the input to the code being tested is too long, the code/test may have to be broken into smaller modules or intermediate points. To elaborate, consider the code flow as:
Input -> A -> B -> C -> Output
// A, B, C are intermediate execution points which are not "publicly accessible".
In the above case, all you can do is
Given Input, check if Output is correct.
Instead, it would be good to expose the intermediate A, B, C, at least to the unit tests, so that you can now break your test into:
Given Input, check if A is correct.
Given A, check if B is correct.
Given B, check if C is correct.
Given C, check if Output is correct.
As you can imagine, if the test fails, it becomes easier to figure out what failed, and hence to fix it.
In C++ how would one test a class constructor? For example:
Class myClass{
int a;
public:
myClass(); //Constructor
void reset(); //resets a to 0;
};
//Constructor defined to initialize a to 0
myClass::myClass(){
reset();
}
//reset sets a to 0
void myClass::reset(){
a = 0;
}
How would I test the myClass() constructor? I can't access the private data member, so would it be valid to say if reset works then the constructor works?
I am going to rephrase your question to: how do I test properties of my class that are not directly observable from my tests because they are protected or private? Here are several possibilities in the order that I usually think about them:
Move the checking part of the test into the class by using an assert(). If what you want to test is that your constructor has successfully established some complex invariant, then you can put an assert() at the end of the constructor that verifies that invariant. Then all your test has to do is to call the constructor - the assert inside the constructor does the actual checking that the test would otherwise have done. If you habitually add asserts as you write the code then that assert will already be there (if asserts make your debug build too slow, use a profiler to pinpoint those asserts that are actually a problem).
Extract the complex piece of code that you want to test into a second class. You can then test the second class through its public interface. This option is attractive when extracting the second class improves the code even when not considering testability.
Change the design of your code to something more testable in some other way. Again, this is attractive when the change is an improvement even when not considering testability.
Strategic retreat. Testing code in isolation is good all other things being equal, but there is no law saying that all code must be tested in isolation. Sometimes the right answer is to just let that code be tested as a side-effect of testing something else. All your other tests for that class will call the constructor, after all, so if something is wrong with the constructor it will probably cause one of those other tests to fail. It may be harder to cover all paths this way and at the time that you get that failure it will be more trouble to track down the cause of the failure because now you do not know if the problem is the constructor or the other thing being tested. That may or may not be a big deal. Is it truly worth the trouble to test this code in isolation, taking into consideration what else you could be using that time on and the probability that this test will ever actually find a bug? Would a code review be a more useful way to spend that time? Is there something else that it would be more useful to do?
If you truly must test something that cannot be tested with assert() and where there is no acceptable refactoring to improve testability, then there are still some options. I've never had to use these for testing, but here they are:
Use a friend declaration to give your tests access to private fields on your class. Each class now needs to maintain a list of friend declarations for tests that need access. This can be a chore to maintain.
Maintain a separate interface for use by tests. Have a set of methods that give the access that you need and whose name has a prefix like "testOnly". So in your case it would be testOnlyGetA(). You will have to be disciplined in only calling these methods from tests and you now need to maintain a larger interface than before.
Hold your nose and do #define private public in your test file before including the header that you want to test.
Upd. Sorry, misunderstood.
You may add a friend testing function to a class or a class-member testing function (which is better to test implementation details, but sometimes is impossible due to testing framework restrictions)
You could add a boost test case to the class:
Class myClass{
private:
int a;
public:
myClass(); //Constructor
void reset(); //resets a to 0;
void test_a_reset() {
BOOST_CHECK_EQUAL(a, 0);
}
};
Im not sure about how you are doing the testing though. This may not be what you want.
You can declare a test class with the same memory layout of your original class, but all members being public.
void testMyClassConstructor()
{
class myClassPublic
{
public:
int a;
myClassPublic();
void reset();
};
typedef union
{
myClass* original;
myClassPublic* testView;
} u_myClass;
myClass testInstance;
u_myClass testUnion;
testUnion.original = &testInstance;
assert(testUnion.testView->a == 0);
}
This is fine in a white box test case, if you can't change the tested class' interface.
However, remember that any changes to the memory layout of the original class will break the unit test aswell.
I have a constructor and a property in the class:
private IMyCollectionObjects _myCollectionObjects;
public MyClassConstructor(string message)
{
_myCollectionObjects = MyCollection.GetCollectionObejects(message);
}
With as much detail can you please help me understand how to unit test this constructor
and GetCollectionObjects method?
How do I completely decouple the
classes? You can give the answer
using any IoC, I want to
understand the concept.
Thank you.
Dependencies on static members such as GetCollectionObjects are difficult to test because you can't replace its implementation at runtime. This means that you cannot fully isolate an instance of MyClassConstructor from the implementation details of GetCollectionObjects. Without isolation of the test target, it can't really be considered a unit test.
See here for further discussion of static dependencies.
You can refactor this code to be fully decoupled (and thus fully testable):
private readonly IMyCollectionObjects _myCollectionObjects;
public MyClassConstructor(IMyCollectionObjects myCollectionObjects)
{
_myCollectionObjects = myCollectionObjects;
}
This bubbles the knowledge of how to turn a message into a collection right out of MyClassConstructor, making the class simpler, more cohesive, and less coupled.
For of all, unit testing is all about unitary testing, that is, one thing at a time.
With as much detail can you please help me understand how to unit test this constructor and GetCollectionObjects method?
First things first, have you unit tested your MyCollection class?
If not, you should begin with it, as your MyClassConstructor class depends on it, that is the basis of dependency injection. Otherwise, how can you manage to know if the results you're getting are right or wrong? You won't be able to test and be sure that it works flawlessly.
How do I completely decouple the classes? You can give the answer using any IoC, I want to understand the concept.
I my humble point of view, you must have a clear reason to make an object dependant of another using dependency injection. Once you make an object depend on another, it makes no sense, in my opinion, to decouple them. One way of decoupling might be to use Unity Application Block of Enterprise Library.
Unit test this constructor
You generally only need to check for three things while testing such a constructor.
That the constructor doesn't return a null value;
That the instance it returns is of the expected type;
That the object you expect to be instantiated through its dependency is actually instiated.
[TestCase("message")]
public void DependentConstructorTest(string message) {
MyClassConstructor myclass = new MyClassConstructor(message);
Assert.IsNotNull(myclass);
Assert.IsInstanceOf(typeof(MyClassConstructor), myclass);
Assert.IsNotNull(myclass.MyCollection); // Where MyCollection represents the property that
// exposes the instance created of the object from
// which your MyClassConstructor class depends on.
}
Note: This test is written using NUnit attributes and assertion methods. Use whatever else you like.
Here is roughly what you would need to do (with some assumptions).
Assuming MyCollection is a static class and GetCollectionObjects parses a string and returns an IMyCollectionObjects, you would first need to make MyCollection non-static, and pass it through the constructor as well. Static classes/methods used in a class create a tight coupling, more or less by definition.
Now you would be constructing the class passing a message string and a MyCollection. Your constructor uses the two in combination to populate a member variable of type IMyCollectionObjects. In order to ensure that this happens as expected, you will need a way to examine the result from outside the class (i.e. a public method). So you will need a property getter that exposes _myCollectionObjects.
Now you just need to call this contructor from one or more tests, and check the property after construction to ensure parsing the string into a collection was successful.
Note that this is really more of an integration test than a discrete unit test. You are really testing that the parsing was successful. If the class represented here is really what you intend to test, the test would really just be checking that GetCollectionObjects was called. The result of that call is really irrelevant, because you would (presumably) have a separate test or set of tests that ensure that the method GetCollectionObjects on MyCollection works as expected.