Kotlintest with Mockk How to clear verify count - unit-testing

So I have the following code:
When("SMS with location update command is received") {
every {
context.getString(R.string.location_sms, any(), any(), any(), any())
} returns "loc"
mainServiceViewModel.handleSms(SmsMessage("123", "location"))
Then("SMS with location is sent to specified phone number") {
verify(exactly = 1) {
smsRepository.sendSms("+123", "loc")
}
}
}
When("motion is detected") {
Then("information SMS is sent to specified phone number") {
verify(exactly = 1) {
smsRepository.sendSms("+123", any())
}
}
}
The problem with it, that both cases pass, even though the second one doesn't do any action. I expect second case to fail, as sendSms method is not even called.
How to reset smsRepository verify count?
How to reset that count before every When case?

This is probably due to the fact that KotlinTest is different from JUnit in what is considered a test and when a Spec instance is created.
The default behavior for KotlinTest is to create a single instance of the Spec per execution. Due to this, your mocks are not getting reset between executions, as you probably have them created at class level.
To fix this, what you can do is either do the mockk inside the test, or change the isolation mode to something that creates the Spec every time a test is executed.
The default isolationMode is IsolationMode.SingleInstance. You can change it on the Spec itself by overriding the isolationMode function:
class MySpec : BehaviorSpec() {
init {
Given("XX") {
Then("YY") // ...
}
}
override fun isolationMode() = IsolationMode.InstancePerTest
}
You can also change it in a ProjectConfig. If you need explanation on how to do it there, check the docs on ProjectConfig
An alternative would be to clear mocks on the afterTest method:
class MySpec : BehaviorSpec() {
init {
Given("XX") {
Then("YY") // ...
}
}
override fun afterTest(testCase: TestCase, result: TestResult) {
clearAllMocks()
}
}
But I'm not sure how that would work in your use-case.

You should try the various clear methods that are provided to reset the state of mocks. Check this related question and MockK's docs for more information.
Check the documentation about test listeners. Basically, every test spec class provides lifecycle methods like beforeEach, which you could override to reset your mocks (using clear). As you are extending BehaviourSpec, you should be able to just override those methods, otherwise confirm how to do this for different testing styles to avoid confusion.

To clear mocks after every test, you can provide a project wide listener:
import io.kotest.core.listeners.TestListener
import io.kotest.core.spec.AutoScan
import io.kotest.core.test.TestCase
import io.kotest.core.test.TestResult
import io.mockk.clearAllMocks
#AutoScan
class MockkClearingTestListener : TestListener {
override suspend fun afterEach(testCase: TestCase, result: TestResult) = clearAllMocks()
}
This works e.g. for every leaf in a WordSpec, and should work for BehaviorSpec as well.

Related

Grails 2.4.4 : How do mock a transient service inside a domain?

I have been trying to figure this out for 2 days now and I am really stuck and frustrated. I have a domain object with a service which is being used for custom validation. The domain looks like this:
class Llama {
String name
transient myFetcherService
static transients = [
'myFetcherService'
]
static constraints = {
name validator: { val, obj ->
if (obj.nameExists(val) == true) {
//return some error here.
}
}
}
protected boolean nameExists(String name) {
List<Llama> llamasList = myFetcherService.fetchExistingLlamasByName(name)
if (llamasList.isEmpty()) {
return false
}
return true
}
}
Now, I have another Service, which simply saves a list of Llama objects. It looks like this:
class LlamaFactoryService {
public void createLlamas(List<String> llamaNames) {
llamaNames.each { name ->
new Llama(name: name).save()
}
}
}
In my test. I keep getting this error:
Failure: createLlamas should create Llammas (com.myLlamaProject.LlamaFactoryServiceSpec)
| java.lang.NullPointerException: Cannot invoke method myFetcherService on null object
I don't understand. In my tests, added a metaClass for the service in the "given" section. When it tries to save, it's telling that the service is null. This is what my test looks like:
given:
def myFetcherService = mockFor(MyFetcherService)
myFetcherService.demand.fetchExistingLlamasByName {def name -> return []}
Llama.metaClass.myFetcherService = myFetcherService.createMock()
when:
service.createLlamas(['Pablo','Juan','Carlos'])
then:
//some validations here....
I also tried using metaClass on the method nameExists() like:
Llama.metaClass.myFetcherService = { def name -> false }
, but it gives me the same nullPointerException as the one above. Could someone point me to the right direction? I'm a bit stuck. :(
Thanks in advance for reading and helping.
You're using a unit test and the general rule for unit tests is that beans generally aren't created for you, so you'll need to inject them yourself.
(Code edited to reflect the fact I misread the question)
I think you want a testing pattern something like:
given:
def mockMyFetcherService = Mock(MyFetcherService) // create the mock
Llama.metaClass.getMyFetcherService = { mockMyFetcherService } // inject the dependency
def returnList = [] // let's just define this here and you can re-use this pattern in other tests more easily
when:
service.createLlamas(['Pablo','Juan','Carlos'])
then:
// tell Spock what you expect to have happen with your Mock - return the List you defined above
3 * mockFetcherService.fetchExistingLlamasByName(_) >> returnList
If the injection of the service into the metaClass doesn't work (suggested here), you could always try using the defineBeans{} closure within the unit test itself (http://www.block-consult.com/blog/2011/08/17/inject-spring-security-service-into-domain-class-for-controller-unit-testing/).
Thus you could try:
defineBeans {
myFetcherService(MockMyFetcherService)
}
where MockMyFetcherService is defined in the same file that defines the test. This is the approach followed here:
See here for examples of more Spock interactions.
If you're using Grails 2.4.3 or below you'll need to put CGLIB in BuildConfig.groovy but I see here that it's already done for you in 2.4.4, so you should be ok just to use Mock(classname).

Issue testing Laravel Controller with Mockery | trying to get property of non-object

I'm very new to testing controllers and I'm running into a problem with a method(). I believe I'm either missing something in my test or my Controller / Repository is designed incorrectly.
The application I'm writing is basically one of those secure "one time" tools. Where you create a note, the system provides you with a URL, once that url is retrieved the note is deleted. I actually have the application written but I am going back to write tests for practice (I know that's backwards).
My Controller:
use OneTimeNote\Repositories\NoteRepositoryInterface as Note;
class NoteController extends \Controller {
protected $note;
public function __construct(Note $note)
{
$this->note = $note;
}
public function getNote($url_id, $key)
{
$note = $this->note->find($url_id, $key);
if (!$note) {
return \Response::json(array('message' => 'Note not found'), 404);
}
$this->note->delete($note->id);
return \Response::json($note);
}
...
I've injected my Note interface in to my controller and all is well.
My Test
use \Mockery as M;
class OneTimeNoteTest extends TestCase {
public function setUp()
{
parent::setUp();
$this->mock = $this->mock('OneTimeNote\Repositories\EloquentNoteRepository');
}
public function mock($class)
{
$mock = M::mock($class);
$this->app->instance($class, $mock);
return $mock;
}
public function testShouldReturnNoteObj()
{
// Should Return Note
$this->mock->shouldReceive('find')->once()->andReturn('test');
$note = $this->call('GET', '/note/1234567890abcdefg/1234567890abcdefg');
$this->assertEquals('test', $note->getContent());
}
}
...
The error I'm getting
1) OneTimeNoteTest::testShouldReturnNoteObj
ErrorException: Trying to get property of non-object
/Users/andrew/laravel/app/OneTimeNote/Controllers/NoteController.php:24
Line 24 is in reference to this line found in my controller:
$this->note->delete($note->id);
Basically my abstracted repository method delete() obviously can't find $note->id because it really doesn't exist in the testing environment. Should I create a Note within the test and try to actually deleting it? Or would that be something that should be a model test? As you can see I need help, thanks!
----- Update -----
I tried to stub the repository to return a Note object as Dave Marshall mentioned in his answer, however I'm now receiving another error.
1) OneTimeNoteTest::testShouldReturnNoteObj
BadMethodCallException: Method Mockery_0_OneTimeNote_Repositories_EloquentNoteRepository::delete() does not exist on this mock object
I do have a delete() method in my repository and I know it's working when I test my route in the browser.
public function delete($id)
{
Note::find($id)->delete();
}
You are stubbing the note repository to return a string, PHP is then trying to retrieve the id attribute of a string, hence the error.
You should stub the repository to return a Note object, something like:
$this->mock->shouldReceive('find')->once()->andReturn(new Note());
Building upon Dave's answer, I was able to figure out what my problem is. I wasn't mocking the delete() method. I didn't understand the need to mock each individual method in my controller that would be called.
I just added this line:
$mock->shouldReceive('delete')->once()->andReturnNull();
Since my delete method is just deleting the note after it is found, I went ahead and mocked it but set it to return null.

Issue when mocking Logger

I am having a strange issue when mocking the log field of a class. Running the same test twice shows an error the second time. This is an example of code:
class AccountConfigJSON {
static Logger log = Logger.getLogger(AccountConfigJSON.class)
def AccountConfigJSON(String jsonString) {
if (jsonString) {
json = new JSONObject(jsonString)
} else {
log.debug("No JSON string for account config. Will not parse")
}
}
}
and this is the specification
class AccountConfigJSONUnitSpec extends UnitSpec {
def loggerMock
def setup(){
loggerMock = Mock(org.apache.log4j.Logger)
org.apache.log4j.Logger.metaClass.static.getLogger = { Class clazz -> loggerMock }
}
def 'If jsonString is null, a log is written'(){
when:
new AccountConfigJSON("")
then:
1 * loggerMock.debug("No JSON string for account config. Will not parse")
}
def 'If jsonString is empty, a log is written'(){
when:
new AccountConfigJSON("")
then:
1 * loggerMock.debug("No JSON string for account config. Will not parse")
}
}
The second test fails showing
| Too few invocations for:
1 * loggerMock.debug("No JSON string for account config. Will not parse") (0 invocations)
but debugging the app using Idea, clearly it runs this sentence. Any idea?
Looks odd that the actual call is executed but the interaction in not recorded. You can get around with it by explicitly assigning the mocked logger to the class as below:
def setup(){
loggerMock = Mock(org.apache.log4j.Logger)
AccountConfigJSON.log = loggerMock
}
From the definition of "interaction", I think the above setup is the best way to go.
Is an Interaction Just a Regular Method Invocation?
Not quite. While an interaction looks similar to a regular method
invocation, it is simply a way to express which method invocations are
expected to occur. A good way to think of an interaction is as a
regular expression that all incoming invocations on mock objects are
matched against. Depending on the circumstances, the interaction may
match zero, one, or multiple invocations.
This only happens while dealing with static object properties in a class. The moment logger is defined non-static in the class under test, everything works as expected without the work around.

How to write a Mockist test of a recursive method

If I have a method that calls itself under a certain condition, is it possible to write a test to verify the behavior? I'd love to see an example, I don't care about the mock framework or language. I'm using RhinoMocks in C# so I'm curious if it is a missing feature of the framework, or if I'm misunderstanding something fundamental, or if it is just an impossibility.
a method that calls itself under a certain condition, is it possible to write a test to verify the behavior?
Yes. However, if you need to test recursion you better separate the entry point into the recursion and the recursion step for testing purposes.
Anyway, here is the example how to test it if you cannot do that. You don't really need any mocking:
// Class under test
public class Factorial
{
public virtual int Calculate(int number)
{
if (number < 2)
return 1
return Calculate(number-1) * number;
}
}
// The helper class to test the recursion
public class FactorialTester : Factorial
{
public int NumberOfCalls { get; set; }
public override int Calculate(int number)
{
NumberOfCalls++;
return base.Calculate(number)
}
}
// Testing
[Test]
public void IsCalledAtLeastOnce()
{
var tester = new FactorialTester();
tester.Calculate(1);
Assert.GreaterOrEqual(1, tester.NumberOfCalls );
}
[Test]
public void IsCalled3TimesForNumber3()
{
var tester = new FactorialTester();
tester.Calculate(3);
Assert.AreEqual(3, tester.NumberOfCalls );
}
Assuming you want to do something like get the filename from a complete path, for example:
c:/windows/awesome/lol.cs -> lol.cs
c:/windows/awesome/yeah/lol.cs -> lol.cs
lol.cs -> lol.cs
and you have:
public getFilename(String original) {
var stripped = original;
while(hasSlashes(stripped)) {
stripped = stripped.substringAfterFirstSlash();
}
return stripped;
}
and you want to write:
public getFilename(String original) {
if(hasSlashes(original)) {
return getFilename(original.substringAfterFirstSlash());
}
return original;
}
Recursion here is an implementation detail and should not be tested for. You really want to be able to switch between the two implementations and verify that they produce the same result: both produce lol.cs for the three examples above.
That being said, because you are recursing by name, rather than saying thisMethod.again() etc., in Ruby you can alias the original method to a new name, redefine the method with the old name, invoke the new name and check whether you end up in the newly defined method.
def blah
puts "in blah"
blah
end
alias blah2 blah
def blah
puts "new blah"
end
blah2
You're misunderstanding the purpose of mock objects. Mocks (in the Mockist sense) are used to test behavioral interactions with dependencies of the system under test.
So, for instance, you might have something like this:
interface IMailOrder
{
void OrderExplosives();
}
class Coyote
{
public Coyote(IMailOrder mailOrder) {}
public void CatchDinner() {}
}
Coyote depends on IMailOrder. In production code, an instance of Coyote would be passed an instance of Acme, which implements IMailOrder. (This might be done through manual Dependency Injection or via a DI framework.)
You want to test method CatchDinner and verify that it calls OrderExplosives. To do so, you:
Create a mock object that implements IMailOrder and create an instance of Coyote (the system under test) by passing the mock object to its constructor. (Arrange)
Call CatchDinner. (Act)
Ask the mock object to verify that a given expectation (OrderExplosives called) was met. (Assert)
When you setup the expectations on the mock object may depend on your mocking (isolation) framework.
If the class or method you're testing has no external dependencies, you don't need (or want) to use mock objects for that set of tests. It doesn't matter if the method is recursive or not.
You generally want to test boundary conditions, so you might test a call that should not be recursive, a call with a single recursive call, and a deeply-recursive call. (miaubiz has a good point about recursion being an implementation detail, though.)
EDIT: By "call" in the last paragraph I meant a call with parameters or object state that would trigger a given recursion depth. I'd also recommend reading The Art of Unit Testing.
EDIT 2: Example test code using Moq:
var mockMailOrder = new Mock<IMailOrder>();
var wily = new Coyote(mockMailOrder.Object);
wily.CatchDinner();
mockMailOrder.Verify(x => x.OrderExplosives());
There isn't anything to monitor stack depth/number of (recursive) function calls in any mocking framework I'm aware of. However, unit testing that the proper mocked pre-conditions provide the correct outputs should be the same as mocking a non-recursive function.
Infinite recursion that leads to a stack overflow you'll have to debug separately, but unit tests and mocks have never gotten rid of that need in the first place.
Here's my 'peasant' approach (in Python, tested, see the comments for the rationale)
Note that implementation detail "exposure" is out of question here, since what you are testing is the underlying architecture which happens to be utilized by the "top-level" code. So, testing it is legitimate and well-behaved (I also hope, it's what you have in mind).
The code (the main idea is to go from a single but "untestable" recursive function to an equivalent pair of recursively dependent (and thus testable) functions):
def factorial(n):
"""Everyone knows this functions contract:)
Internally designed to use 'factorial_impl' (hence recursion)."""
return factorial_impl(n, factorial_impl)
def factorial_impl(n, fct=factorial):
"""This function's contract is
to return 'n*fct(n-1)' for n > 1, or '1' otherwise.
'fct' must be a function both taking and returning 'int'"""
return n*fct(n - 1) if n > 1 else 1
The test:
import unittest
class TestFactorial(unittest.TestCase):
def test_impl(self):
"""Test the 'factorial_impl' function,
'wiring' it to a specially constructed 'fct'"""
def fct(n):
"""To be 'injected'
as a 'factorial_impl''s 'fct' parameter"""
# Use a simple number, which will 'show' itself
# in the 'factorial_impl' return value.
return 100
# Here we must get '1'.
self.assertEqual(factorial_impl(1, fct), 1)
# Here we must get 'n*100', note the ease of testing:)
self.assertEqual(factorial_impl(2, fct), 2*100)
self.assertEqual(factorial_impl(3, fct), 3*100)
def test(self):
"""Test the 'factorial' function"""
self.assertEqual(factorial(1), 1)
self.assertEqual(factorial(2), 2)
self.assertEqual(factorial(3), 6)
The output:
Finding files...
['...py'] ... done
Importing test modules ... done.
Test the 'factorial' function ... ok
Test the 'factorial_impl' function, ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK

Groovy: Verify construction of stubbed URL

The test class below verifies that a simple HttpService gets content from a given URL. Both the implementations shown make the test pass, though one is clearly wrong because it constructs the URL with an incorrect argument.
To avoid this and correctly specify the behaviour I want, I'd like to verify that in the use block of the test case, I construct one (and only one) instance of the URL class, and that the url argument to the constructor is correct. A Groovy enhancement seems like it would let me add the statement
mockURLContext.demand.URL { assertEquals "http://www.foo.com", url }
but what can I do without that Groovy enhancement?
Update: Replaced "mock" with "stub" in the title, as I'm only interested in checking the state not necessarily the detail of the interactions. Groovy has a StubFor mechanism that I haven't used, so I'll leave my code as is, but I think you could just replace MockFor with StubFor throughout.
import grails.test.*
import groovy.mock.interceptor.MockFor
class HttpServiceTests extends GrailsUnitTestCase {
void testGetsContentForURL() {
def content = [text : "<html><body>Hello, world</body></html>"]
def mockURLContext = new MockFor(URL.class)
mockURLContext.demand.getContent { content }
mockURLContext.use {
def httpService = new HttpService()
assertEquals content.text, httpService.getContentFor("http://www.foo.com")
}
}
}
// This is the intended implementation.
class HttpService {
def getContentFor(url) {
new URL(url).content.text
}
}
// This intentionally wrong implementation also passes the test!
class HttpService {
def getContentFor(url) {
new URL("http://www.wrongurl.com").content.text
}
}
What does mocking the URL get you? It makes the test difficult to write. You won't be able to react to feedback the mock objects give you about the design of the API of the URL class, because it's not under your control. And if you don't precisely fake the behaviour of the URL and what it exposes about the HTTP protocol, the test will not be reliable.
You want to test that your "HttpService" object actually loads the data correctly from a given URL, copes with different content type encodings correctly, handles different classes of HTTP status code appropriately, and so forth. When I need to test this kind object -- one that merely wraps some underlying technical infrastructure -- I write a real integration test that verifies that the object really does use the underlying technology correctly.
For HTTP I write a test that creates an HTTP server, plugs a servlet into the server that will return some canned data, passed the URL of the servlet to the object to make it load the data, check that the loaded result is the same as the canned data used to initialise the servlet, and stop the server in the fixture tear-down. I use Jetty or the simple HTTP server that is bundled with JDK 6.
I'd only use mock objects to test the behaviour of objects that talk to the interface(s) of that object I've integration tested.
Putting on my "Programming in the Small" and "Unit test 100%" hat, you could consider this as a single method that does too many things. You could refactor the HttpService to:
class HttpService {
def newURLFrom(urlString) {
new URL(urlString)
}
def getContentText(url) {
url.content.text
}
def getContentFor(urlString) {
getContentText(newURLFrom(urlString))
}
}
This would give you a few more options for testing, as well as split out the factory aspect from the property manipulation. The testing options are bit more mundane then:
class HttpServiceTests extends GroovyTestCase {
def urlString = "http://stackoverflow.com"
def fauxHtml = "<html><body>Hello, world</body></html>";
def fauxURL = [content : [text : fauxHtml]]
void testMakesURLs() {
assertEquals(urlString,
new HTTPService().newURLFrom(urlString).toExternalForm())
}
void testCanDeriveContentText() {
assertEquals(fauxHtml, new HTTPService().getContentText(fauxURL));
}
// Going a bit overboard to test the line combining the two methods
void testGetsContentForURL() {
def service = new HTTPService()
def emc = new ExpandoMetaClass( service.class, false )
emc.newURLFrom = { input -> assertEquals(urlString, input); return fauxURL }
emc.initialize()
service.metaClass = emc
assertEquals(fauxHtml, service.getContentFor(urlString))
}
}
I think that this makes all the assertions that you want, but at the cost of sacrificing test readability in the last case.
I would agree with Nat about this making more sense as an integration test. (You are integrating with Java's URL library on some level.) But assuming that this example simplifies some complex logic, you could use the metaclass to override the instances class effictvely partially mocking the instance.
It's tough to mock out JDK classes that are declared final... Your problem, as you reference through the enhancement, is that there is no way to create a URL other than calling the constructor.
I try to separate the creation of these kinds of objects from the rest of my code; I'd create a factory to separate the creation the URLs. This should be simple enough to not warrant a test. Others take a typical wrapper/decorator approach. Or you may be able to apply the adapter pattern to translate to domain objects that you write.
Here is a similar answer to a surprisingly similar problem: Mocking a URL in Java
I think this demonstrates something that a lot of people learn after doing more testing: the code we write to make things more testable is meant to isolate what we desire to test from what we can safely say is already tested somewhere else. It's a fundamental assumption we have to make in order to do unit testing. It can also provide a decent example of why good unit tests aren't necessarily about 100% code coverage. They have to be economical, too.
Hope this helps.
What exactly are you expecting to have fail? It is not readily apparent what you are trying to test with that code. By Mocking URL.getContent you are telling Groovy to always return the variable content when URL.getContent() is invoked. Are you wishing to make the return value of URL.getContent() conditional based upon the URL string? If that is the case, the following accomplishes that:
import grails.test.*
import groovy.mock.interceptor.MockFor
class HttpServiceTests extends GrailsUnitTestCase {
def connectionUrl
void testGetsContentForURL() {
// put the desired "correct" URL as the key in the next line
def content = ["http://www.foo.com" : "<html><body>Hello, world</body></html>"]
def mockURLContext = new MockFor(URL.class)
mockURLContext.demand.getContent { [text : content[this.connectionUrl]] }
mockURLContext.use {
def httpService = new HttpService()
this.connectionUrl = "http://www.wrongurl.com"
assertEquals content.text, httpService.getContentFor(this.connectionUrl)
}
}
}