The test class below verifies that a simple HttpService gets content from a given URL. Both the implementations shown make the test pass, though one is clearly wrong because it constructs the URL with an incorrect argument.
To avoid this and correctly specify the behaviour I want, I'd like to verify that in the use block of the test case, I construct one (and only one) instance of the URL class, and that the url argument to the constructor is correct. A Groovy enhancement seems like it would let me add the statement
mockURLContext.demand.URL { assertEquals "http://www.foo.com", url }
but what can I do without that Groovy enhancement?
Update: Replaced "mock" with "stub" in the title, as I'm only interested in checking the state not necessarily the detail of the interactions. Groovy has a StubFor mechanism that I haven't used, so I'll leave my code as is, but I think you could just replace MockFor with StubFor throughout.
import grails.test.*
import groovy.mock.interceptor.MockFor
class HttpServiceTests extends GrailsUnitTestCase {
void testGetsContentForURL() {
def content = [text : "<html><body>Hello, world</body></html>"]
def mockURLContext = new MockFor(URL.class)
mockURLContext.demand.getContent { content }
mockURLContext.use {
def httpService = new HttpService()
assertEquals content.text, httpService.getContentFor("http://www.foo.com")
}
}
}
// This is the intended implementation.
class HttpService {
def getContentFor(url) {
new URL(url).content.text
}
}
// This intentionally wrong implementation also passes the test!
class HttpService {
def getContentFor(url) {
new URL("http://www.wrongurl.com").content.text
}
}
What does mocking the URL get you? It makes the test difficult to write. You won't be able to react to feedback the mock objects give you about the design of the API of the URL class, because it's not under your control. And if you don't precisely fake the behaviour of the URL and what it exposes about the HTTP protocol, the test will not be reliable.
You want to test that your "HttpService" object actually loads the data correctly from a given URL, copes with different content type encodings correctly, handles different classes of HTTP status code appropriately, and so forth. When I need to test this kind object -- one that merely wraps some underlying technical infrastructure -- I write a real integration test that verifies that the object really does use the underlying technology correctly.
For HTTP I write a test that creates an HTTP server, plugs a servlet into the server that will return some canned data, passed the URL of the servlet to the object to make it load the data, check that the loaded result is the same as the canned data used to initialise the servlet, and stop the server in the fixture tear-down. I use Jetty or the simple HTTP server that is bundled with JDK 6.
I'd only use mock objects to test the behaviour of objects that talk to the interface(s) of that object I've integration tested.
Putting on my "Programming in the Small" and "Unit test 100%" hat, you could consider this as a single method that does too many things. You could refactor the HttpService to:
class HttpService {
def newURLFrom(urlString) {
new URL(urlString)
}
def getContentText(url) {
url.content.text
}
def getContentFor(urlString) {
getContentText(newURLFrom(urlString))
}
}
This would give you a few more options for testing, as well as split out the factory aspect from the property manipulation. The testing options are bit more mundane then:
class HttpServiceTests extends GroovyTestCase {
def urlString = "http://stackoverflow.com"
def fauxHtml = "<html><body>Hello, world</body></html>";
def fauxURL = [content : [text : fauxHtml]]
void testMakesURLs() {
assertEquals(urlString,
new HTTPService().newURLFrom(urlString).toExternalForm())
}
void testCanDeriveContentText() {
assertEquals(fauxHtml, new HTTPService().getContentText(fauxURL));
}
// Going a bit overboard to test the line combining the two methods
void testGetsContentForURL() {
def service = new HTTPService()
def emc = new ExpandoMetaClass( service.class, false )
emc.newURLFrom = { input -> assertEquals(urlString, input); return fauxURL }
emc.initialize()
service.metaClass = emc
assertEquals(fauxHtml, service.getContentFor(urlString))
}
}
I think that this makes all the assertions that you want, but at the cost of sacrificing test readability in the last case.
I would agree with Nat about this making more sense as an integration test. (You are integrating with Java's URL library on some level.) But assuming that this example simplifies some complex logic, you could use the metaclass to override the instances class effictvely partially mocking the instance.
It's tough to mock out JDK classes that are declared final... Your problem, as you reference through the enhancement, is that there is no way to create a URL other than calling the constructor.
I try to separate the creation of these kinds of objects from the rest of my code; I'd create a factory to separate the creation the URLs. This should be simple enough to not warrant a test. Others take a typical wrapper/decorator approach. Or you may be able to apply the adapter pattern to translate to domain objects that you write.
Here is a similar answer to a surprisingly similar problem: Mocking a URL in Java
I think this demonstrates something that a lot of people learn after doing more testing: the code we write to make things more testable is meant to isolate what we desire to test from what we can safely say is already tested somewhere else. It's a fundamental assumption we have to make in order to do unit testing. It can also provide a decent example of why good unit tests aren't necessarily about 100% code coverage. They have to be economical, too.
Hope this helps.
What exactly are you expecting to have fail? It is not readily apparent what you are trying to test with that code. By Mocking URL.getContent you are telling Groovy to always return the variable content when URL.getContent() is invoked. Are you wishing to make the return value of URL.getContent() conditional based upon the URL string? If that is the case, the following accomplishes that:
import grails.test.*
import groovy.mock.interceptor.MockFor
class HttpServiceTests extends GrailsUnitTestCase {
def connectionUrl
void testGetsContentForURL() {
// put the desired "correct" URL as the key in the next line
def content = ["http://www.foo.com" : "<html><body>Hello, world</body></html>"]
def mockURLContext = new MockFor(URL.class)
mockURLContext.demand.getContent { [text : content[this.connectionUrl]] }
mockURLContext.use {
def httpService = new HttpService()
this.connectionUrl = "http://www.wrongurl.com"
assertEquals content.text, httpService.getContentFor(this.connectionUrl)
}
}
}
Related
Usually I was ending up writing test cases for a Domain by writing them for constraints and any custom methods(created by us in application) as we know we shouldn't test obvious.
But the time we started using coverage plugin, we found that our domains line of code is not fully covered which was due to gorm hooks(onInsert, beforeUpdate) that we never wrote test cases for.
Is there a way we can test these. One possible way that seems obvious but not suitable is to call another method(containing all code which was earlier in hooks) within these hooks and test that method only and be carefree for hooks.
Any solutions...
Edit
Sample code in domain that I want to unit-test:
class TestDomain{
String activationDate
def beforeInsert() {
this.activationDate = (this.activationDate) ?: new Date()//first login date would come here though
encodePassword()
}
}
How can I unit-test beforeInsert or I would end up writing integration test case?
Perhaps a unit test like:
import grails.test.mixin.TestFor
#TestFor(TestDomain)
class TestDomainSpec extends Specification {
def "test beforeSave"() {
given:
mockForConstraintsTests(TestDomain)
when:
def testDomain = new TestDomain().save(flush:true)
then:
testDomain.activationDate != null
}
}
I've just started with unit testing in CakePHP (yay!) and ran into the following challenge. Hope somebody can help me :-)
Situation
My model uses a Behavior to send changes to an API after saving it locally. I would like to fake all calls made to the API during the test (those will be tested seperately) to save load on the API server, and more important, not actually save the changes :-)
I'm using CakePHP 2.4.1.
What I've tried
Read the docs. The manual shows how to do this for Components and Helpers but not for Behaviors.
Google. What I've found:
A Google Group post which says it simply "isn't possible". I don't take no for an answer.
An article explaining how to mock an object. Comes pretty close.
The code from the article reads:
$provider = $this->getMock('OurProvider', array('getInfo'));
$provider->expects($this->any())
->method('getInfo')
->will($this->returnValue('200'));
It might be the wrong direction, but I think that might be a good start.
What I want
Effectively: A snippet of code to demo how to mock a behavior in a CakePHP Model for unit testing purposes.
Maybe this question will result in an addition of the CakePHP manual too as an added bonus, since I feel it's missing in there.
Thanks in advance for the effort!
Update (2013-11-07)
I've found this related question, which should answer this question (partly). No need to mock up the API, instead I can create a Behavior test that the model will use.
I'm trying to figure out what that BehaviorTest should look like.
Use the class registry
As with many classes, behaviors are added to the class registry using the class name as the key, and for subsequent requests for the same object loaded from the classregistry. Therefore, the way to mock a behavior is simply to put it in the class registry before using it.
Full Example:
<?php
App::uses('AppModel', 'Model');
class Example extends AppModel {
}
class TestBehavior extends ModelBehavior {
public function foo() {
throw new \Exception('Real method called');
}
}
class BehaviorExampleTest extends CakeTestCase {
/**
* testNormalBehavior
*
* #expectedException Exception
* #expectedExceptionMessage Real method called
* #return void
*/
public function testNormalBehavior() {
$model = ClassRegistry::init('Example');
$model->Behaviors->attach('Test');
$this->assertInstanceOf('TestBehavior', $model->Behaviors->Test);
$this->assertSame('TestBehavior', get_class($model->Behaviors->Test));
$this->assertSame(['foo' => ['Test', 'foo']], $model->Behaviors->methods());
$model->foo();
}
public function testMockedBehavior() {
$mockedBehavior = $this->getMock('TestBehavior', ['foo', 'bar']);
ClassRegistry::addObject('TestBehavior', $mockedBehavior);
$model = ClassRegistry::init('Example');
$model->Behaviors->attach('Test');
$this->assertInstanceOf('TestBehavior', $model->Behaviors->Test);
$this->assertNotSame('TestBehavior', get_class($model->Behaviors->Test));
$expected = [
'foo' => ['Test', 'foo'],
'bar' => ['Test', 'bar'],
'expects' => ['Test', 'expects'], // noise, due to being a mock
'staticExpects' => ['Test', 'staticExpects'], // noise, due to being a mock
];
$this->assertSame($expected, $model->Behaviors->methods());
$model->foo(); // no exception thrown
$mockedBehavior
->expects($this->once())
->method('bar')
->will($this->returnValue('something special'));
$return = $model->bar();
$this->assertSame('something special', $return);
}
}
Say I've got a controller with an action that receives two parameters.
It invokes two services, one with each parameter, the services both return strings
each of those strings are passed as arguments to a template
the result is passed to Ok and returned.
I want to write a simple unit test that ensures:
1 - The correct services are invoked with the correct parameters
2 - The return values from the services are passed to the correct attributes of the template
What is the best way to do that?
Using Mockito with Specs2, I mock services to verify their method calls.
My controller is instantiated by Spring. That allows me to treat it is as a class instead of object. => That is essential to make controller testable. Here an example:
#Controller
class MyController #Autowired()(val myServices: MyServices) extends Controller
To enable Spring for controllers, you have to define a Global object, as the Play! documentation explains:
object Global extends GlobalSettings {
val context = new ClassPathXmlApplicationContext("application-context.xml")
override def getControllerInstance[A](controllerClass: Class[A]): A = {
context.getBean(controllerClass)
}
}
My unit test doesn't need Spring; I just pass collaborators (mocks) to constructor.
However, concerning the rendered template, I test only for the type of result (Ok, BadRequest, Redirection etc...).
Indeed, I noticed it's not easy at all to make my test scan the whole rendered template in details (parameters sent to it etc..), with only unit testing.
Thus, in order to assert that the right template is called with the right arguments, I trust my acceptance tests running Selenium, or a possible functional test, if you prefer, to scan for the whole expected result.
2 - The return values from the services are passed to the correct
attributes of the template
It's pretty easy to check for that..How? By trusting compiler! Prefer to pass some custom types to your template instead of simple primitives for instance:
phone: String would become: phone: Phone. (a simple value object).
Therefore, no fear to pass the attributes in a non-expected order to your template (in unit test or real production code). Compiler indeed will warn.
Here's an example of one of my unit test (simplified) using specs2:
(You will note the use of a wrapper: WithFreshMocks).
This case class would allow to refresh all variables (mocks in this case) test after test.
Thus a good way to reset mocks.
class MyControllerSpec extends Specification with Mockito {
def is =
"listAllCars should retrieve all cars" ! WithFreshMocks().listAllCarsShouldRetrieveAllCars
case class WithFreshMocks() {
val myServicesMock = mock[MyServices]
val myController = new MyController(myServicesMock)
def listAllCarsShouldRetrieveAllCars = {
val FakeGetRequest = FakeRequest() //fakeRequest needed by controller
mockListAllCarsAsReturningSomeCars()
val result = myController.listAllCars(FakeGetRequest).asInstanceOf[PlainResult] //passing fakeRequest to simulate a true request
assertOkResult(result).
and(there was one(myServicesMock).listAllCars()) //verify that there is one and only one call of listAllCars. If listAllCars would take any parameters that you expected to be called, you could have precise them.
}
private def mockListAllCarsAsReturningSomeCars() {
myServicesMock.listAllCars() returns List[Cars](Car("ferrari"), Car("porsche"))
}
private def assertOkResult(result: PlainResult) = result.header.status must_== 200
}
So, I came up with a cake pattern and mockito based solution:
given the service:
trait Service {
def indexMessage : String
}
trait ServiceImpl {
def indexMessage = {
"Hello world"
}
}
Then the controller looks like:
object Application extends ApplicationController
with ServiceImpl {
def template = views.html.index.apply
}
trait ApplicationController extends Controller
with Service {
def template: (String) => play.api.templates.Html
def index = Action {
Ok(template("controller got:" + indexMessage))
}
}
And the test looks like:
class ApplicationControllerSpec extends Specification with Mockito {
"cake ApplicationController" should {
"return OK with the results of the service invocation" in {
val expectedMessage = "Test Message"
val m = mock[(String) => play.api.templates.Html]
object ApplicationControllerSpec extends ApplicationController {
def indexMessage = expectedMessage
def template = m
}
val response = ApplicationControllerSpec.index(FakeRequest())
status(response) must equalTo(OK)
there was one(m).apply(Matchers.eq("controller got:" + expectedMessage))
}
}
}
I had a lot of trouble getting Mockito working.
It requires an extra dependency and I had a lot of trouble working out how to use the matchers in scala (I'm quite comfortable using it in java)
Ultimately I think the above answer is better, avoid using String and other primitive types where you can wrap them in task specific types, then you get compiler warnings.
Also I would generally avoid doing things like the "controller got:" prefixing in the controller.
It's there in this case so I can verify that it went through, in the real world that should be done by some other component (controllers are just for plumbing IMO)
and thank you in advance for any and all your assistance.
I have a method that I'm trying to test.
Within this method is a call to UserMembership.Validate()
//custom override but the code isn't functional yet and is outside the scope of the test.
I want to therefore mock (using moq) the return result so that the actual test of the method can succeed.
Here is the code
public LoginResponse Login(LoginRequest request)
{
var response = new LoginResponse(request.RequestId);
// Validate client tag and access token
if (!ValidateRequest(request, response, Validate.ClientTag | Validate.AccessToken))
return response;
if (!UserMembership.ValidateUser(request.UserName, request.Password))
{
response.Acknowledge = AcknowledgeType.Failure;
response.Messages = "Invalid username and/or password.";
//response.MessageCode = -4;
return response;
}
_userName = request.UserName;
return response;
}
So, my test is for LoginResponse() but I want to 'fake' the UserMembership return value (bool) to true...
Simple enough I'm sure for you guys.
TIA, Hugh.
You could probably re-title your question to "How do you use a mocking framework with unit testing 99% of the time," because you're right on track for doing just that - a very typical usage.
You're going to want to extract an interface from your UserMembership class (right click inside the class, select "refactor" and then "extract interface."), then use Moq to create mock instances of that interface for use within your tests. Then you can use Moq to "setup" the behavior of that mock to do anything you want it to during your test. The syntax would look like this:
var userMembershipMock = new Mock<IUserMembership>();
userMembershipMock.Setup(m=> m.ValidateUser(It.Is<string>(str=> str == "myUserName"), It.Is<string>(str=> str == "myPassword"))).Returns(true);
Then you would create a new instance of your class, passing in your mock instance of IUserMembership (but since you'll make your class's constructor takes an argument of the interface type, your class won't care whether you're passing it a mock or an actual UserMembership instance
MyClass myClass = new MyClass(userMembershipMock.Object);
after which you could begin actually testing the behavior of your MyClass:
var request = new LoginRequest { UserName = "myUserName", Password = "myPassword" };
LoginResponse response = myClass.Login(request);
And then you can assert that your class's response is what you expect:
Assert.AreEqual(AcknowledgeType.Success, response.Acknowledge);
or you can verify that your mock's method (or property) was invoked as you expected:
userMembershipMock.Verify(m=> m.ValidateUser(It.Is<string>(str=> str == "myUserName"), It.Is<string>(str=> str == "myPassword")), Times.Once());
and so on.
The Moq quick start page is kind of sort of a one-page read, and can teach you 99% of everything that you need to know to use it.
The only way I can think of to mock UserMembership in this case (assuming it's not a property) is to use an IoC framework like Castle Windsor or Ninject. When you use an IoC container you would refactor your calls to UserMembership into an interface (IUserMembership) and use the container to provide an implementation:
if (Container.Resolve<IUserMembership>().ValidateUser(request.UserName, request.Password))
Then in your unit test Setup you would register the implementation of IUserMembership to be the mock object:
var mock = new Mock<IUserMembership>();
Container.Register<IUserMemberhip>().Instance(mock.Object);
You would have to also create a production implementation. If this is the standard UserMembership class, this implementation will probably do nothing other than UserMembership. Although, there are other ways to mimic this kind of duck typing.
If I have a method that calls itself under a certain condition, is it possible to write a test to verify the behavior? I'd love to see an example, I don't care about the mock framework or language. I'm using RhinoMocks in C# so I'm curious if it is a missing feature of the framework, or if I'm misunderstanding something fundamental, or if it is just an impossibility.
a method that calls itself under a certain condition, is it possible to write a test to verify the behavior?
Yes. However, if you need to test recursion you better separate the entry point into the recursion and the recursion step for testing purposes.
Anyway, here is the example how to test it if you cannot do that. You don't really need any mocking:
// Class under test
public class Factorial
{
public virtual int Calculate(int number)
{
if (number < 2)
return 1
return Calculate(number-1) * number;
}
}
// The helper class to test the recursion
public class FactorialTester : Factorial
{
public int NumberOfCalls { get; set; }
public override int Calculate(int number)
{
NumberOfCalls++;
return base.Calculate(number)
}
}
// Testing
[Test]
public void IsCalledAtLeastOnce()
{
var tester = new FactorialTester();
tester.Calculate(1);
Assert.GreaterOrEqual(1, tester.NumberOfCalls );
}
[Test]
public void IsCalled3TimesForNumber3()
{
var tester = new FactorialTester();
tester.Calculate(3);
Assert.AreEqual(3, tester.NumberOfCalls );
}
Assuming you want to do something like get the filename from a complete path, for example:
c:/windows/awesome/lol.cs -> lol.cs
c:/windows/awesome/yeah/lol.cs -> lol.cs
lol.cs -> lol.cs
and you have:
public getFilename(String original) {
var stripped = original;
while(hasSlashes(stripped)) {
stripped = stripped.substringAfterFirstSlash();
}
return stripped;
}
and you want to write:
public getFilename(String original) {
if(hasSlashes(original)) {
return getFilename(original.substringAfterFirstSlash());
}
return original;
}
Recursion here is an implementation detail and should not be tested for. You really want to be able to switch between the two implementations and verify that they produce the same result: both produce lol.cs for the three examples above.
That being said, because you are recursing by name, rather than saying thisMethod.again() etc., in Ruby you can alias the original method to a new name, redefine the method with the old name, invoke the new name and check whether you end up in the newly defined method.
def blah
puts "in blah"
blah
end
alias blah2 blah
def blah
puts "new blah"
end
blah2
You're misunderstanding the purpose of mock objects. Mocks (in the Mockist sense) are used to test behavioral interactions with dependencies of the system under test.
So, for instance, you might have something like this:
interface IMailOrder
{
void OrderExplosives();
}
class Coyote
{
public Coyote(IMailOrder mailOrder) {}
public void CatchDinner() {}
}
Coyote depends on IMailOrder. In production code, an instance of Coyote would be passed an instance of Acme, which implements IMailOrder. (This might be done through manual Dependency Injection or via a DI framework.)
You want to test method CatchDinner and verify that it calls OrderExplosives. To do so, you:
Create a mock object that implements IMailOrder and create an instance of Coyote (the system under test) by passing the mock object to its constructor. (Arrange)
Call CatchDinner. (Act)
Ask the mock object to verify that a given expectation (OrderExplosives called) was met. (Assert)
When you setup the expectations on the mock object may depend on your mocking (isolation) framework.
If the class or method you're testing has no external dependencies, you don't need (or want) to use mock objects for that set of tests. It doesn't matter if the method is recursive or not.
You generally want to test boundary conditions, so you might test a call that should not be recursive, a call with a single recursive call, and a deeply-recursive call. (miaubiz has a good point about recursion being an implementation detail, though.)
EDIT: By "call" in the last paragraph I meant a call with parameters or object state that would trigger a given recursion depth. I'd also recommend reading The Art of Unit Testing.
EDIT 2: Example test code using Moq:
var mockMailOrder = new Mock<IMailOrder>();
var wily = new Coyote(mockMailOrder.Object);
wily.CatchDinner();
mockMailOrder.Verify(x => x.OrderExplosives());
There isn't anything to monitor stack depth/number of (recursive) function calls in any mocking framework I'm aware of. However, unit testing that the proper mocked pre-conditions provide the correct outputs should be the same as mocking a non-recursive function.
Infinite recursion that leads to a stack overflow you'll have to debug separately, but unit tests and mocks have never gotten rid of that need in the first place.
Here's my 'peasant' approach (in Python, tested, see the comments for the rationale)
Note that implementation detail "exposure" is out of question here, since what you are testing is the underlying architecture which happens to be utilized by the "top-level" code. So, testing it is legitimate and well-behaved (I also hope, it's what you have in mind).
The code (the main idea is to go from a single but "untestable" recursive function to an equivalent pair of recursively dependent (and thus testable) functions):
def factorial(n):
"""Everyone knows this functions contract:)
Internally designed to use 'factorial_impl' (hence recursion)."""
return factorial_impl(n, factorial_impl)
def factorial_impl(n, fct=factorial):
"""This function's contract is
to return 'n*fct(n-1)' for n > 1, or '1' otherwise.
'fct' must be a function both taking and returning 'int'"""
return n*fct(n - 1) if n > 1 else 1
The test:
import unittest
class TestFactorial(unittest.TestCase):
def test_impl(self):
"""Test the 'factorial_impl' function,
'wiring' it to a specially constructed 'fct'"""
def fct(n):
"""To be 'injected'
as a 'factorial_impl''s 'fct' parameter"""
# Use a simple number, which will 'show' itself
# in the 'factorial_impl' return value.
return 100
# Here we must get '1'.
self.assertEqual(factorial_impl(1, fct), 1)
# Here we must get 'n*100', note the ease of testing:)
self.assertEqual(factorial_impl(2, fct), 2*100)
self.assertEqual(factorial_impl(3, fct), 3*100)
def test(self):
"""Test the 'factorial' function"""
self.assertEqual(factorial(1), 1)
self.assertEqual(factorial(2), 2)
self.assertEqual(factorial(3), 6)
The output:
Finding files...
['...py'] ... done
Importing test modules ... done.
Test the 'factorial' function ... ok
Test the 'factorial_impl' function, ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK