I'm quite new to unit testing. I've read around on here as well as done some Googling, but am still a bit confused as to the meaning of each of the four meanings. I came across the following definition which help....
Stub - stubs are minimal implementations of interfaces or base classes
Spy - a spy will record which members were invoked
Fake - more complex, a fake may resemble a production implementation
Mock - A mock is usually dynamically created by a mock library and depending on its configuration, a mock can behave like a dummy, a stub, or a spy
However, I'd like to simplify the meaning (if possible) and ask a few questions.
Do all of the above only relate to functions, or can they be objects or any other type?
Stub - Is Stubbing bascially a way to provide dummy info instead of making the calling to the actual database? So for example, if I had an API call, instead of actually making a call to the API, I just instead make a GET request to a JSON file which is in my tests folder which has some dummy data in, and use that instead of making the API call?
Spy - so is this a way of tracking what happens to a function for example. Meaning you follow when it's called, where it gets passed around to?
Fake - Is this for example a function which you create inside the test file to mimic the real function or be a simpified version of the actual function?
Thanks in advance.
There are multiple attempts at definitions. To my knowledge there is no fully consistent definition, probably due to the fact that mocking frameworks defines things slightly differently. Martin Fowler lists the following (https://martinfowler.com/bliki/TestDouble.html):
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an InMemoryTestDatabase is a good example).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test.
Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent.
Mocks are pre-programmed with expectations which form a specification of the calls they are expected to receive. They can throw an exception if they receive a call they don't expect and are checked during verification to ensure they got all the calls they were expecting.
Fowler has them from this page: http://xunitpatterns.com/Test%20Double.html where you can read about them in a bit more detail.
No, they apply to more then just functions
Yes
Kind of, in most cases you will spy on an object
Yes
The implementation and nomenclatur will depend of what framework you use.
Related
I am trying to wrap my head around the practice of unit testing. I have read that unit tests should not be dependent on the implementations.
The way I understand this is, for example, if we have a function int addTwoNumbers (int a, int b) we should be testing whether, for example addition return correct results (i.e. addTwoNumbers (2, 2) == 4)) and we should not care whether, for example, addTwoNumbers calls operator + just once - it might as well use bit manipulation for this.
It seemed reasonable for me and - in my opinion - offers a decent decoupling between the tests and the code.
Enter the mocking frameworks. As far I can tell from reading their documentation, their functionality amounts to
(a) generating objects that implement placeholder functionality of some interfaces of base classes (stubbing) and
(b) checking whether that functionality was called according to expectations set forth by the tester.
I have no problems with (a) - I understand that we sometimes need to hard-code some functionality of external dependencies for the testing. I do not understand though, why should we check whether the tested code called mock's functionality in a way expected by the tester.
Shouldn't we be interested only in what a tested method returns or how it modifies its out arguments and not really care about its implementation details? Does the verification functionality of the mocking frameworks not introduce tight coupling between tested and testing code?
I do not understand though, why should we check whether the tested code called mock's functionality in a way expected by the tester
Because the contract of a method is not always to return something or to modify its arguments. Sometimes the contract of the method is (or includes) to have side effects. Think for example of the following method:
void notifyServerOfError(error: string) {
this.http.post('/api/errors', {
error: error,
ip: myIpAddress
});
}
This method doesn't return anything. It doesn't modify its arguments. Its sole responsibility is to send a specific object, containing specific details, to a specific URL. And unit-testing this method should thus verify that the contract is respected.
A good way to do that is to mock the http dependency and to check that, when this method is called, its post() method indeed being called with the correct URL and the correct data.
I know that many mocking libraries let the programmer check that a method was called only once. But why is that useful?
Also, why is it useful to verify the parameters to a mock's method?
When doing unit tests you are testing a isolated method: all other methods are supposed to work correctly, and you test only that your method behaves in the expected (specified...) way.
But in many occasions the expected way implies calling methods of classes you depend on (via dependency injection, if you want to do unit testing).
For these reason you need to assure that this calls are really done... and of course they are called with the expected parameters.
Example:
In your real application you have a repository class that stores all your changes in the database (and does only this!). But for unit test your "business" class (where all your business rules are defined), you should mock that "repository" class: then you must check that this mocked class receives the correct updated data.
In general, the answers to both of those questions are the same.
It's useful if the requirements of the unit/method you're testing specify that that behavior is required. If that behavior is required, then that's what you need to verify is actually happening.
If it's important to ensure that a particular method is only called once, then you can do that. If it doesn't matter that a method is called more than once, then don't test for it explicitly. Note that the default for the Mockito "verify" method is "times(1)", which means that it confirms that the method was called once and only once.
Concerning the second question, you might want to verify the parameters if it's entirely possible the method could be called with different parameters, and you don't want to count those occurrences, you only care about a specific set of parameter values.
I have a repository with a method called ConvertToBusinessEntity which converts the data from the data source to a business object. This method is supposed to be used by other methods in the repository namely the Get, GetAll, etc.
This method is unit tested. I check if the data is being retrieved correctly from the data source and the values are being put in the entity correct properties.
But now I need to test and create the GetEntity method which is supposed to call ConvertToBusinessEntity. The logic behind ConvertToBusiness is tested. I just need to know how to verify that a method in the class being tested is called (not a dependency).
Does anyone know how to do this or any alternative method?
I thought of an alternative method but I am not sure if it's the best. I could extend the class under test and have a counter increasing each time the method is called.
Another one would be to mock the class itself.
What do you think? Any other suggestions?
Having ConvertToBusinessEntity in repository is not very good idea. Responsibility of repository is working with data store (CRUD). Mapping of data types is responsibility of some mapper class. Otherwise your repository has too many responsibilities. Take a look on what you are trying to test:
I check if the data is being retrieved correctly from the data source
and the values are being put in the entity correct properties
You see this and? Your test can fail on two completely different reasons. Also you should change repository on two completely different reasons. Best approach here is persisting business entities directly. Modern ORMs allows doing that without polluting business entity with attributes or forcing it inheriting some data-access specific class.
If you really want to have data mapping logic in repository, then make it private (actually only repository should require conversion of business entity to some data-access object) and don't care how this logic is implemented. Let it be part of internal class implementation. You should care only about repository being able to accept or return filled business entities - that's the responsibility of repository. It doesn't matter how mapping is implemented in repository. You should test what repository does, instead of how. So just check that expected business objects are returned by repository.
I just need to know how to verify that a method in the class being tested is called (not a dependency).
But do you really need to do that? If your GetEntity method operates correctly, do you really care how it operates? Do you really care if it performs its function by delegating to ConvertToBusiness, or by some other means?
I recommend instead that you
Think of each method as having a specification.
That specification describes what the outputs and publicly visible manipulations it must make. That do not describe how a method performs its function; that is an implementation detail that could change.
Your unit tests check only that your methods conform to their specification.
You might nevertheless use your knowledge about the implementation to choose good test cases.
But, you might declare, if I do that I am not unit testing my method code; my test of GetEntity depends on both the GetEntity method and the ConvertToBusiness method: two units, so an integration test rather than a unit test. But do you mock the methods of the runtime environment? Of course not. The line between unit and integration testing is not so clear.
More philosophically, you can not create good mock objects in many cases. The reason is that, for most methods, the manner in which an object delegates to associated objects is undefined. Whether it does delegate, and how, is left by the specification as an implementation detail. The only requirement is that, on delegating, the method satisfies the preconditions of its delegate. In such a situation, only a fully functional (non-mock) delegate will do. If the real object checks its preconditions, failure to satisfy a precondition on delegating will cause a test failure. And debugging that test failure will be easy.
I'm testing code that uses a third-party PBKDF2class to make a hash. I'm mocking this object in my tests. This class takes a number of iterations as an argument, which increases the time it takes to hash. Is it a good idea to add an assertion inside the mock class to check that the correct number of iterations is being passed in, or should I add a piece of data to the fake hash it returns, so that my actual unit test can check the iteration count?
Indeed, it is a good idea to add assertion in mocked classes, although that isn't what most people would call it. That is how mock objects should work, they should both return data that the class being tested needs and verify that the class being tested is calling the right function and sending the right information.
However, don't mock these classes manually yourself, that'll take forever for all of the cases you need. Use a unit test library with strong mocking abilities, like the x-unit library (which is available for many languages, php-unit, j-unit, etc). These libraries allow you to easily mock an object, set up expectations and return values without you having to make hundreds of different mocks manually.
If the expectations aren't met on the mocked classes, the test will fail letting you know something went wrong.
I have just started to read Professional Test Driven Development with C#: Developing Real World Applications with TDD
I have a hard time understanding stubs, fakes and mocks. From what I understand so far, they are fake objects used for the purpose of unit testing your projects, and that a mock is a stub with conditional logic into it.
Another thing I think I have picked up is that mocks are somehow related with dependency injection, a concept which I only managed to understand yesterday.
What I do not get is why I would actually use them. I cannot seem to find any concrete examples online that explains them properly.
Can anyone please explain to me this concepts?
As I've read in the past, here's what I believe each term stands for
Stub
Here you are stubbing the result of a method to a known value, just to let the code run without issues. For example, let's say you had the following:
public int CalculateDiskSize(string networkShareName)
{
// This method does things on a network drive.
}
You don't care what the return value of this method is, it's not relevant. Plus it could cause an exception when executed if the network drive is not available. So you stub the result in order to avoid potential execution issues with the method.
So you end up doing something like:
sut.WhenCalled(() => sut.CalculateDiskSize()).Returns(10);
Fake
With a fake you are returning fake data, or creating a fake instance of an object. A classic example are repository classes. Take this method:
public int CalculateTotalSalary(IList<Employee> employees) { }
Normally the above method would be passed a collection of employees that were read from a database. However in your unit tests you don't want to access a database. So you create a fake employees list:
IList<Employee> fakeEmployees = new List<Employee>();
You can then add items to fakeEmployees and assert the expected results, in this case the total salary.
Mocks
When using mock objects you intend to verify some behaviour, or data, on those mock objects. Example:
You want to verify that a specific method was executed during a test run, here's a generic example using Moq mocking framework:
public void Test()
{
// Arrange.
var mock = new Mock<ISomething>();
mock.Expect(m => m.MethodToCheckIfCalled()).Verifiable();
var sut = new ThingToTest();
// Act.
sut.DoSomething(mock.Object);
// Assert
mock.Verify(m => m.MethodToCheckIfCalled());
}
Hopefully the above helps clarify things a bit.
EDIT:
Roy Osherove is a well-known advocate of Test Driven Development, and he has some excellent information on the topic. You may find it very useful :
http://artofunittesting.com/
They are all variations of the Test Double. Here is a very good reference that explains the differences between them: http://xunitpatterns.com/Test%20Double.html
Also, from Martin Fowler's post: http://martinfowler.com/articles/mocksArentStubs.html
Meszaros uses the term Test Double as the generic term for any kind of
pretend object used in place of a real object for testing purposes.
The name comes from the notion of a Stunt Double in movies. (One of
his aims was to avoid using any name that was already widely used.)
Meszaros then defined four particular kinds of double:
Dummy objects: are passed around but never actually used. Usually they
are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes
them not suitable for production (an in memory database is a good
example).
Stubs provide canned answers to calls made during the test,
usually not responding at all to anything outside what's programmed in
for the test. Stubs may also record information about calls, such as
an email gateway stub that remembers the messages it 'sent', or maybe
only how many messages it 'sent'.
Mocks are what we are talking about here: objects pre-programmed with expectations which form a
specification of the calls they are expected to receive.
Of these kinds of doubles, only mocks insist upon behavior verification. The
other doubles can, and usually do, use state verification. Mocks
actually do behave like other doubles during the exercise phase, as
they need to make the SUT believe it's talking with its real
collaborators.
This PHP Unit's manual helped me a lot as introduction:
"Sometimes it is just plain hard to test the system under test (SUT) because it depends on other components that cannot be used in the test environment. This could be because they aren't available, they will not return the results needed for the test or because executing them would have undesirable side effects. In other cases, our test strategy requires us to have more control or visibility of the internal behavior of the SUT." More: https://phpunit.de/manual/current/en/test-doubles.html
And i find better "introductions" when looking for "test doubles" as mocks, fakes, stubs and the others are known.