I know that many mocking libraries let the programmer check that a method was called only once. But why is that useful?
Also, why is it useful to verify the parameters to a mock's method?
When doing unit tests you are testing a isolated method: all other methods are supposed to work correctly, and you test only that your method behaves in the expected (specified...) way.
But in many occasions the expected way implies calling methods of classes you depend on (via dependency injection, if you want to do unit testing).
For these reason you need to assure that this calls are really done... and of course they are called with the expected parameters.
Example:
In your real application you have a repository class that stores all your changes in the database (and does only this!). But for unit test your "business" class (where all your business rules are defined), you should mock that "repository" class: then you must check that this mocked class receives the correct updated data.
In general, the answers to both of those questions are the same.
It's useful if the requirements of the unit/method you're testing specify that that behavior is required. If that behavior is required, then that's what you need to verify is actually happening.
If it's important to ensure that a particular method is only called once, then you can do that. If it doesn't matter that a method is called more than once, then don't test for it explicitly. Note that the default for the Mockito "verify" method is "times(1)", which means that it confirms that the method was called once and only once.
Concerning the second question, you might want to verify the parameters if it's entirely possible the method could be called with different parameters, and you don't want to count those occurrences, you only care about a specific set of parameter values.
Related
I'm quite new to unit testing. I've read around on here as well as done some Googling, but am still a bit confused as to the meaning of each of the four meanings. I came across the following definition which help....
Stub - stubs are minimal implementations of interfaces or base classes
Spy - a spy will record which members were invoked
Fake - more complex, a fake may resemble a production implementation
Mock - A mock is usually dynamically created by a mock library and depending on its configuration, a mock can behave like a dummy, a stub, or a spy
However, I'd like to simplify the meaning (if possible) and ask a few questions.
Do all of the above only relate to functions, or can they be objects or any other type?
Stub - Is Stubbing bascially a way to provide dummy info instead of making the calling to the actual database? So for example, if I had an API call, instead of actually making a call to the API, I just instead make a GET request to a JSON file which is in my tests folder which has some dummy data in, and use that instead of making the API call?
Spy - so is this a way of tracking what happens to a function for example. Meaning you follow when it's called, where it gets passed around to?
Fake - Is this for example a function which you create inside the test file to mimic the real function or be a simpified version of the actual function?
Thanks in advance.
There are multiple attempts at definitions. To my knowledge there is no fully consistent definition, probably due to the fact that mocking frameworks defines things slightly differently. Martin Fowler lists the following (https://martinfowler.com/bliki/TestDouble.html):
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an InMemoryTestDatabase is a good example).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test.
Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent.
Mocks are pre-programmed with expectations which form a specification of the calls they are expected to receive. They can throw an exception if they receive a call they don't expect and are checked during verification to ensure they got all the calls they were expecting.
Fowler has them from this page: http://xunitpatterns.com/Test%20Double.html where you can read about them in a bit more detail.
No, they apply to more then just functions
Yes
Kind of, in most cases you will spy on an object
Yes
The implementation and nomenclatur will depend of what framework you use.
I have a repository with a method called ConvertToBusinessEntity which converts the data from the data source to a business object. This method is supposed to be used by other methods in the repository namely the Get, GetAll, etc.
This method is unit tested. I check if the data is being retrieved correctly from the data source and the values are being put in the entity correct properties.
But now I need to test and create the GetEntity method which is supposed to call ConvertToBusinessEntity. The logic behind ConvertToBusiness is tested. I just need to know how to verify that a method in the class being tested is called (not a dependency).
Does anyone know how to do this or any alternative method?
I thought of an alternative method but I am not sure if it's the best. I could extend the class under test and have a counter increasing each time the method is called.
Another one would be to mock the class itself.
What do you think? Any other suggestions?
Having ConvertToBusinessEntity in repository is not very good idea. Responsibility of repository is working with data store (CRUD). Mapping of data types is responsibility of some mapper class. Otherwise your repository has too many responsibilities. Take a look on what you are trying to test:
I check if the data is being retrieved correctly from the data source
and the values are being put in the entity correct properties
You see this and? Your test can fail on two completely different reasons. Also you should change repository on two completely different reasons. Best approach here is persisting business entities directly. Modern ORMs allows doing that without polluting business entity with attributes or forcing it inheriting some data-access specific class.
If you really want to have data mapping logic in repository, then make it private (actually only repository should require conversion of business entity to some data-access object) and don't care how this logic is implemented. Let it be part of internal class implementation. You should care only about repository being able to accept or return filled business entities - that's the responsibility of repository. It doesn't matter how mapping is implemented in repository. You should test what repository does, instead of how. So just check that expected business objects are returned by repository.
I just need to know how to verify that a method in the class being tested is called (not a dependency).
But do you really need to do that? If your GetEntity method operates correctly, do you really care how it operates? Do you really care if it performs its function by delegating to ConvertToBusiness, or by some other means?
I recommend instead that you
Think of each method as having a specification.
That specification describes what the outputs and publicly visible manipulations it must make. That do not describe how a method performs its function; that is an implementation detail that could change.
Your unit tests check only that your methods conform to their specification.
You might nevertheless use your knowledge about the implementation to choose good test cases.
But, you might declare, if I do that I am not unit testing my method code; my test of GetEntity depends on both the GetEntity method and the ConvertToBusiness method: two units, so an integration test rather than a unit test. But do you mock the methods of the runtime environment? Of course not. The line between unit and integration testing is not so clear.
More philosophically, you can not create good mock objects in many cases. The reason is that, for most methods, the manner in which an object delegates to associated objects is undefined. Whether it does delegate, and how, is left by the specification as an implementation detail. The only requirement is that, on delegating, the method satisfies the preconditions of its delegate. In such a situation, only a fully functional (non-mock) delegate will do. If the real object checks its preconditions, failure to satisfy a precondition on delegating will cause a test failure. And debugging that test failure will be easy.
I'm testing code that uses a third-party PBKDF2class to make a hash. I'm mocking this object in my tests. This class takes a number of iterations as an argument, which increases the time it takes to hash. Is it a good idea to add an assertion inside the mock class to check that the correct number of iterations is being passed in, or should I add a piece of data to the fake hash it returns, so that my actual unit test can check the iteration count?
Indeed, it is a good idea to add assertion in mocked classes, although that isn't what most people would call it. That is how mock objects should work, they should both return data that the class being tested needs and verify that the class being tested is calling the right function and sending the right information.
However, don't mock these classes manually yourself, that'll take forever for all of the cases you need. Use a unit test library with strong mocking abilities, like the x-unit library (which is available for many languages, php-unit, j-unit, etc). These libraries allow you to easily mock an object, set up expectations and return values without you having to make hundreds of different mocks manually.
If the expectations aren't met on the mocked classes, the test will fail letting you know something went wrong.
Suppose I have several unit tests in a test class ([TestClass] in VSUnit in my case). I'm trying to test just one thing in each test (doesn't mean just one Assert though). Imagine there's one test (e.g. Test_MethodA() ) that tests a method used in other tests as well. I do not want to put an assert on this method in other tests that use it to avoid duplicity/maintainability issues so I have the assert in only this one test. Now when this test fails, all tests that depend on correct execution of that tested method fail as well. I want to be able to locate the problem faster so I want to be somehow pointed to Test_MethodA. It would e.g. help if I could make some of the tests in the test class execute in a particular order and when they fail I'd start looking for cause of the failure in the first failing test. Do you have any idea how to do this?
Edit: By suggesting that a solution would be to execute the tests in a particular order I have probably went too far and in the wrong direction. I don't care about the order of the tests. It's just that some of the tests will always fail if a prequisite isn't valid. E.g. I have a test class that tests a DAO class (ok, probably not a UNIT test, but there's logic in the database stored procedures that needs to be tested but that's not the point here I think). I need to insert some records into a table in order to test that a method responsible for retrieving the records (let's call it GetAll()) gets them all in the correct order e.g. I do the insert by using a method on the DAO class. Let's call it Insert(). I have tests in place that verify that the Insert() method works as expected. Now I want to test the GetAll() method. In order to get the database in a desired state I use the Insert() method. If Insert() doesn't work, most tests for GetAll() will fail. I'd prefer to mark the tests that can't pass because Insert() doesn't work as inconclusive rather than failed. It would ease finding the cause of the problem if I know which method/test to look into first.
You can't (and shouldn't) execute unit tests in a specific order. The underlying reason for this is to prevent Interacting Tests - I realize that your motivation for requesting such a feature is different, but that's the reason why unit test frameworks don't allow you to order tests. In fact, last time I checked, xUnit.net even randomizes the order.
One could argue that the fact that some of your tests depend on a different method call on the same class is a symptom of tight coupling, but that's not always the case (state machines come to mind).
However, if possible, consider using a Back Door instead of the other method in question.
If you can't do either that or decouple the interdependency (e.g. by making the first method virtual and using the Extract and Override technique), you will have to live with it.
Here's an example:
public class MyClass
{
public virtual void FirstMethod() { // do something... }
public void SecondMethod() {}
}
Since FirstMethod is virtual, you can derive from MyClass and override its behavior. You can also use a dynamic mock to do that for you. With Moq, it would look like this:
var sutStub = new Mock<MyClass>();
// by default, Moq overrides all virtual methods without calling base
// Now invoke both methods in sequence:
sutStub.Object.FirstMethod(); // overriden by Moq, so it does nothing
sutSutb.Object.SecondMethod();
I think I would indeed have the assertion on the method_A() result in every tests relying on its result, even if this introduces some duplication. Then I would use the assertion message to point to the method_A() failure.
assert("method_A() returned true", true, rc);
Perhaps will I end extracting the method_A() call and the assertion into an helper function to remove the duplication.
Now let's imagine method_A() queries an object and returns it, or NULL when no object is found. Then this assertion is a guard ; and it is necessary with languages suchas C, C++ that do not have NullPointerException.
I'm afraid you can't do this. The only solution is to redesign your code and break it up into smaller methods so that unit tests can call these one by one. Of course this isn't always desirable.
With Visual Studio you can order your tests: see here. But I'd like to advise you to stay away from this technique as much as possible: unit tests are meant to be run anywhere, anytime and in every order.
EDIT: why is this a problem for you? All failing tests point to the same method anyway...
If a function just calls another function or performs actions. How do I test it? Currently, I enforce all the functions should return a value so that I could assert the function return values. However, I think this approach mass up the API because in the production code. I don't need those functions to return value. Any good solutions?
I think mock object might be a possible solution. I want to know when should I use assert and when should I use mock objects? Is there any general guide line?
Thank you
Let's use BufferedStream.Flush() as an example method that doesn't return anything; how would we test this method if we had written it ourselves?
There is always some observable effect, otherwise the method would not exist. So the answer can be to test for the effect:
[Test]
public void FlushWritesToUnderlyingStream()
{
var memory = new byte[10];
var memoryStream = new MemoryStream(memory);
var buffered = new BufferedStream(memoryStream);
buffered.Write(0xFF);
Assert.AreEqual(0x00, memory[0]); // not yet flushed, memory unchanged
buffered.Flush();
Assert.AreEqual(0xFF, memory[0]); // now it has changed
}
The trick is to structure your code so that these effects aren't too hard to observe in a test:
explicitly pass collaborator objects,
just like how the memoryStream is passed
to the BufferedStream in the constructor.
This is called dependency
injection.
program against an interface, just
like how BufferedStream is programmed
against the Stream interface. This enables
you to pass simpler, test-friendly implementations (like MemoryStream in this case) or use a mocking framework (like MoQ or RhinoMocks), which is all great for unit testing.
Sorry for not answering straight but ... are you sure you have the exact balance in your testing?
I wonder if you are not testing too much ?
Do you really need to test a function that merely delegates to another?
Returns only for the tests
I agree with you when you write you don't want to add return values that are useful only for the tests, not for production. This clutters your API, making it less clear, which is a huge cost in the end.
Also, your return value could seem correct to the test, but nothing says that the implementation is returning the return value that corresponds to the implementation, so the test is probably not proving anything anyway...
Costs
Note that testing has an initial cost, the cost of writing the test.
If the implementation is very easy, the risk of failure is ridiculously low, but the time spend testing still accumulates (over hundred or thousands cases, it ends up being pretty serious).
But more than that, each time you refactor your production code, you will probably have to refactor your tests also. So the maintenance cost of your tests will be high.
Testing the implementation
Testing what a method does (what other methods it calls, etc) is critized, just like testing a private method... There are several points made:
this is fragile and costly : any code refactoring will break the tests, so this increases the maintenance cost
Testing a private method does not bring much safety to your production code, because your production code is not making that call. It's like verifying something you won't actually need.
When a code delegates effectively to another, the implementation is so simple that the risk of mistakes is very low, and the code almost never changes, so what works once (when you write it) will never break...
Yes, mock is generally the way to go, if you want to test that a certain function is called and that certain parameters are passed in.
Here's how to do it in Typemock (C#):
Isolate.Verify.WasCalledWithAnyArguments(()=> myInstance.WeatherService("","", null,0));
Isolate.Verify.WasCalledWithExactArguments(()=> myInstance. StockQuote("","", null,0));
In general, you should use Assert as much as possible, until when you can't have it ( For example, when you have to test whether you call an external Web service API properly, in this case you can't/ don't want to communicate with the web service directly). In this case you use mock to verify that a certain web service method is correctly called with correct parameters.
"I want to know when should I use assert and when should I use mock objects? Is there any general guide line?"
There's an absolute, fixed and important rule.
Your tests must contain assert. The presence of assert is what you use to see if the test passed or failed. A test is a method that calls the "component under test" (a function, an object, whatever) in a specific fixture, and makes specific assertions about the component's behavior.
A test asserts something about the component being tested. Every test must have an assert, or it isn't a test. If it doesn't have assert, it's not clear what you're doing.
A mock is a replacement for a component to simplify the test configuration. It is a "mock" or "imitation" or "false" component that replaces a real component. You use mocks to replace something and simplify your testing.
Let's say you're going to test function a. And function a calls function b.
The tests for function a must have an assert (or it's not a test).
The tests for a may need a mock for function b. To isolate the two functions, you test a with a mock for function b.
The tests for function b must have an assert (or it's not a test).
The tests for b may not need anything mocked. Or, perhaps b makes an OS API call. This may need to be mocked. Or perhaps b writes to a file. This may need to be mocked.