Unit test logger messages using specs2 + scalalogging - unit-testing

I have a little problem here and really have no idea how to implement unit testing for logger messages. Of course, it sounds a little weird, but for me it's really interesting topic. But let me be more specific.
I have some scala class and test specification:
class Testable extends Logging {
def method() = {
// some method calls
logger.info("Message1")
}
}
class TestableSpec extends Specification with ShouldMatchers with Mockito {
"Testable instance" should {
// some important tests
"print proper log message during method call" in {
// And how to test that logger really prints proper message ("Message1")?
}
}
}
My first thought was to intercept underlying logger engine messages but it seems a little hard thing to implement due to usage of mixins in Testable class, therefore any ideas to do such things would be very helpful.
UPDATE:
I finally implemented a test and decided to share my solution with community. We cannot mock scalalogging.Logger class directly because it's final but we still can mock underlying slf4j Logger. To clarify an idea:
class Testable extends Logging {
def foo() = {
// ...
logger.info("Foo has been called")
}
}
// Another imports are omitted.
import com.typesafe.scalalogging.slf4j.Logger
import org.slf4j.{Logger => Underlying}
class TestableSpec extends Specification with Mockito with ShouldMatchers {
def initTestable(mocked: Underlying): Testable = {
new Testable() {
override lazy val logger = Logger(mocked)
}
}
"Testable instance" should {
"invoke logger with a proper message" in {
val mocked = mock[Underlying]
mocked.isInfoEnabled returns true // Should be set to true for test
initTestable(mocked).foo()
there was one(mocked).info("Foo has been called")
}
}
}
Thanks Eric for his help. His answer was a key to the solution.

One possibility is to use Mockito to check method calls:
class Testable extends Logging {
def method() = {
// some method calls
logger.info("Message1")
}
}
class TestableSpec extends Specification with ShouldMatchers with Mockito {
"Testable instance" should {
"print proper log message during method call" in {
val mockLogger = mock[Logger]
val testable = new Testable {
// switch the logger with a mock instance
override val logger = mockLogger
}
testable.method()
there was one(mockLogger).info("Message1")
}
}
}
This is the main idea but you might have to adapt it depending on your exact traits and logging framework:
logger must be overridable
the info method must not be final (one of Mockito's limitations)

Good question... and good answer ! I had some trouble with the Mockito mixin. So I am using Eric's approach with the Java DSL for Mockito. If anyone is interested in this variation, here is the slightly modified code:
import com.typesafe.scalalogging.{LazyLogging, Logger, StrictLogging}
import org.mockito.Mockito
import org.mockito.Mockito._
import org.slf4j.{Logger => Underlying}
class Testable extends LazyLogging {
def foo() = {
logger.info("Foo has been called")
}
}
import org.junit.runner.RunWith
import org.scalatest.{BeforeAndAfterEach, FunSuite}
import org.scalatest.junit.JUnitRunner
import org.scalatest.matchers.ShouldMatchers
#RunWith(classOf[JUnitRunner])
class LoggerTest
extends FunSuite with ShouldMatchers with BeforeAndAfterEach {
def initTestable(mocked: Underlying): Testable = {
new Testable() {
override lazy val logger = Logger(mocked)
}
}
test("the mockito stuff") {
val mocked = Mockito.mock(classOf[Underlying])
when(mocked.isInfoEnabled()).thenReturn(true)
initTestable(mocked).foo()
verify(mocked).info("Foo has been called")
}
}

Related

Spock unit testing assert log calls and see output

I am using spock to test Java Spring Boot code. It gets a logback logger over the lombok #Slf4j annotation.
Dummy class with log call
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
#Slf4j
#Component
public class Clazz {
public void method() {
// ... code
log.warn("message", new RuntimeException());
}
}
The Spock Spec
import groovy.util.logging.Slf4j
import org.junit.Rule
import org.slf4j.Logger
import spock.lang.Specification
#Slf4j
class LogSpec extends Specification {
Clazz clazz = new Clazz()
private Logger logger = Mock(Logger.class)
#Rule
ReplaceSlf4jLogger replaceSlf4jLogger = new ReplaceSlf4jLogger(Clazz, logger)
def "warning ia logged"() {
given: "expected message"
when: "when calling the method"
clazz.method()
then: "a warning is logged"
1 * logger.warn(_, _) >> {
msg, ex -> log.warn(msg, ex)
}
}
}
Helper to switch the real with the mock logger taken from this answer.
import org.junit.rules.ExternalResource
import org.slf4j.Logger
import java.lang.reflect.Field
import java.lang.reflect.Modifier
/**
* Helper to exchange loggers set by lombok with mock logger
*
* allows to assert log action.
*
* Undos change after test to keep normal logging in other tests.
*
* code from this answer answer
*/
class ReplaceSlf4jLogger extends ExternalResource {
Field logField
Logger logger
Logger originalLogger
ReplaceSlf4jLogger(Class logClass, Logger logger) {
logField = logClass.getDeclaredField("log")
this.logger = logger
}
#Override
protected void before() throws Throwable {
logField.accessible = true
Field modifiersField = Field.getDeclaredField("modifiers")
modifiersField.accessible = true
modifiersField.setInt(logField, logField.getModifiers() & ~Modifier.FINAL)
originalLogger = (Logger) logField.get(null)
logField.set(null, logger)
}
#Override
protected void after() {
logField.set(null, originalLogger)
}
}
I would like to test log calls, but still see the log message.
I am using the solution from this answer, it works for the assertion but I don't see the log because it is a mock call.
I came up with this solution, which does a the call with the logger of the groovy spec.
1 * logger.warn(_ , _) >> {
msg, ex -> log.warn(msg, ex)
}
But I find it verbose, any idea how I could create a helper function for it. I am not very familiar with functional groovy and moving this code into a function is not working.
I also tried a Spy instead of a Mock but that gets me an error because the logger class is final.
import ch.qos.logback.classic.Logger
private Logger logger = Spy(Logger.class)
>> org.spockframework.mock.CannotCreateMockException: Cannot create mock
for class ch.qos.logback.classic.Logger because Java mocks cannot mock final classes.
If the code under test is written in Groovy, use a Groovy mock.
Logger class at runtime
package ch.qos.logback.classic;
public final class Logger implements org.slf4j.Logger, LocationAwareLogger, AppenderAttachable<ILoggingEvent>, Serializable {
Thanks
Actually in your MCVE you expect the warn(_, _) method to be called with two parameters, but you are not logging like that in Clazz, so either you have to change Clazz to also log an exception or change the test to expect a method call with one parameter. I am doing the latter here.
As for your problem, the solution is to not use a mock but a spy. You need to tell Spock which exact class you want to spy on, though. This is because you cannot spy on an interface type, of course. I have chosen a SimpleLogger (change to whatever you use in your application).
package de.scrum_master.stackoverflow
import groovy.util.logging.Slf4j
import org.junit.Rule
import org.slf4j.impl.SimpleLogger
import spock.lang.Specification
#Slf4j
class LombokSlf4jLogTest extends Specification {
SimpleLogger logger = Spy(constructorArgs: ["LombokSlf4jLogTest"])
#Rule
ReplaceSlf4jLogger replaceSlf4jLogger = new ReplaceSlf4jLogger(Clazz, logger)
def "warning is logged"() {
when: "when calling the method"
new Clazz().method()
then: "a warning is logged"
1 * logger.warn(_)
}
}
Update: For what it is worth, here is a version which also works with LogBack-Classic instead of Log4J-Simple on the classpath. Instead of directly spying on the final class, let's just spy on a Groovy #Delegate:
Please also note that I changed to *_ in the test so as to accommodate to warn calls with an arbitrary number of arguments.
package de.scrum_master.stackoverflow
import groovy.util.logging.Slf4j
import org.junit.Rule
import org.slf4j.Logger
import spock.lang.Specification
#Slf4j
class LombokSlf4jLogTest extends Specification {
def logger = Spy(new LoggerDelegate(originalLogger: log))
#Rule
ReplaceSlf4jLogger replaceSlf4jLogger = new ReplaceSlf4jLogger(Clazz, logger)
def "warning is logged"() {
when: "when calling the method"
new Clazz().method()
then: "a warning is logged"
1 * logger.warn(*_)
true
}
static class LoggerDelegate {
#Delegate Logger originalLogger
}
}
Update 2020-01-23: I just found this one again and noticed that I forgot to explain why the #Delegate solution works: because a Groovy delegate automatically implements all interfaces which the class of the delegate instance also implements by default. In this case the logger field is declared as Logger which is an interface type. This is also why e.g. Log4J or Logback instances can be used based on the configuration. The trick of mocking or spying on a final class type not implementing an interface or used explicitly with its class name would not work in that case because the delegating class would not (and could not) be a subclass of the final class type and thus could not be injected instead of the delegate.
Update 2020-04-14: I did not mention before that if you don't want to spy on a real logger but simply use a dummy you can check interactions on, just use a regular Spock mock on the org.slf4j.Logger interface: def logger = Mock(Logger) That is actually the simplest solution and you don't clutter your test log with exception stack traces and other log output. I was so focused on helping the OP with his spy solution that I did not mention this before.
These is one more "creative" approach for this kind of issue I would like to share.
Instead of mocking the logger you can create an "artificial" appender, add it programmatically to the logger in the class under-test.
The appender will keep track of the logged messages and during the verification phase you will get those logged message and verify
You'll end up with something like this (pseudo code just to show the idea):
class MsgTrackingAppender implements Appender { // Appender of your logging system
private List<LogEvent> events = new ArrayList<>();
public void doAppend(LogEvent evt) {
events.add(evt);
}
public List<LogEvent> getEvents() {
return events;
}
}
// now in test you can do:
class LogSpec extends Specification {
def "test me"() {
given:
Clazz underTest = Clazz()
MsgTrackingAppender appender = new MsgTrackingAppender()
LogFactory.getLogger(Clazz.class).addAppender(appender)
when:
underTest.method()
then:
appender.events.size == 1
appender.events[0].level == Level.WARN
appender.events[0].message == ... // verify whatever you want on the messages
}
}
IMO this approach is easier to use than extensive mocking but its a matter of taste of course.

How can I setup test data for functional testing in Grails?

So we have a restful service that we want to test using a restclient in grails.
The test code should go something like this...
class MyControllerSpec extends Specification {
def setup() {
this.dbEntity = new DbEntity("someid123").save();
}
void "Test entity GET"{
given:
RestBuilder rest = new RestBuilder()
when: "The DB entity service is hit"
RestResponse restResponse = rest.post("http://localhost:8080/api/someentity/$id");
then: "A 200 error is sent"
restResponse.status == 200
}
The problem I am having is the setup method blows up on .save() because there is not hibernate session. How can I manipulate my database before running a test?
You can define a method named like "setupData", and call it in the "given" block of "Test entity GET" testcase.
def setupData() { this.dbEntity = new DbEntity("someid123").save(); }
If you need to load some data before each funcional test, you can create a helper class, with #Shared variables or methods or both. Even you could override the setup, setupSpec methods in that class.
Your first class does not extends Specification now, DataLoader class (helper class) instead.
class MyControllerSpec extends DataLoader {
void setup(){
createEntity()
}
void "Test entity GET"{
given:
RestBuilder rest = new RestBuilder()
when: "The DB entity service is hit"
RestResponse restResponse = rest.post("http://localhost:8080/api/someentity/$dbEntity.id");
then: "A 200 error is sent"
restResponse.status == 200
}
}
And your helper class is the one which extends Specification, with its methods and #Shared variables.
import spock.lang.Shared
class DataLoader extends Specification {
#Shared DbEntity dbEntity
void createEntity(){
dbEntity = new DbEntity("someid123").save();
}
}
When extending GebSpec in Grails 2.5.6, none of the other answers helped: I would still get
Method on class [...] was used outside of a Grails application
on the save() call.
Adding #TestFor(DbEntity) to the test class helped.
NB: While that annotation breaks integration tests, it seems to be necessary here. Not sure why that is.
You probably want to use the remote-control plugin. In Grails 2.x, add this to your BuildConfig.groovy:
repositories {
...
mavenRepo "http://dl.bintray.com/alkemist/maven/"
}
plugins {
...
test ":remote-control:2.0"
}
After refreshing dependencies and potentially adjusting some settings (see e.g. here and here), you can use it like so in tests:
// <project>/test/functional/<package>/MyControllerSpec.groovy
class MyControllerSpec extends GebSpec {
RemoteControl remote
DbEntity dbEntity
def setup() {
this.remote = new RemoteControl()
this.dbEntity = remote {
new DbEntity("someid123").save()
}
}
def cleanup() {
remote {
DbEntity.findAll().each { dbe ->
println("Deleting $dbe")
dbe.delete()
}
}
}
Note:
You can invoke remote in given/setup and when blocks as well.
Sometimes, it seems to be necessary to wrap core in remote { ... } in a DbEntity.withTransaction { ... }. Maybe that's obvious for the more intitiated; I stumbled over that.
If you want to return a DbEntity from remote it must be serializable.

mocking a function call

Using Spock framework for java unit testing.
When unit testing a method method1() and method1 is calling a method method2(),In method2() having a code statement as below :
Config config = new Config();
TimeZone tz=TimeZone.getTimeZone(config.getProps().getProperty(Constants.SERVER_TIMEZONE));
The call config.getProps().getProperty(Constants.SERVER_TIMEZONE)
returns America/Cambridge_Bay
In getProps method the property files is fetched from weblogic domain and it will not be available in spcok, its taking path as null.
Please suggest how can this function call be mocked in spock.
We can use meta class injection to mock response method2 in unit testing given block. Here ClassName is the class to which method2 belongs to.
ClassName.metaClass.method2 = { inputParameters ->
return "America/Cambridge_Bay"
}
Also it is advisable to use #ConfineMetaClass([ClassName]) annotation on the unit test to confine the meta class injection changes to your test case.
Lets start with an example which simulates your situation:
class Config {
Properties getProps() {
def props = new Properties()
props.setProperty(Constants.SERVER_TIMEZONE, 'America/Cambridge_Bay')
props
}
}
class Constants {
static String SERVER_TIMEZONE = 'TIMEZONE'
}
Config config = new Config()
def timeZoneID = config.getProps().getProperty(Constants.SERVER_TIMEZONE)
def tz = TimeZone.getTimeZone(timeZoneID)
assert tz.ID == 'America/Cambridge_Bay'
Since method2() doesn't get a Config instance injected into it, mocks are out of the question. So we'll use Groovy's metaClass, at the class level (since instance level is out of the question too, for the same reason). You can override Config.getProps() like this:
Config.metaClass.getProps {
def props = new Properties()
props.setProperty(Constants.SERVER_TIMEZONE, 'Etc/UTC')
props
}
So you can write your Spock test roughly like this:
// import Constants
// import Config class
class FooSpec extends Specification {
#ConfineMetaClassChanges
def "test stuff"() {
when:
Config.metaClass.getProps {
def props = new Properties()
props.setProperty(Constants.SERVER_TIMEZONE, 'America/Cambridge_Bay')
props
}
// Do more stuff
then:
// Check results
}
}
PS
If you can modify method2() to have Config injected, that would be preferable since then you can use Groovy's MockFor.

Unit testing of a class with StaticLoggerBinder

I do have a simple class like this:
package com.example.howtomocktest
import groovy.util.logging.Slf4j
import java.nio.channels.NotYetBoundException
#Slf4j
class ErrorLogger {
static void handleExceptions(Closure closure) {
try {
closure()
}catch (UnsupportedOperationException|NotYetBoundException ex) {
log.error ex.message
} catch (Exception ex) {
log.error 'Processing exception {}', ex
}
}
}
And I would like to write a test for it, here is a skeleton:
package com.example.howtomocktest
import org.slf4j.Logger
import spock.lang.Specification
import java.nio.channels.NotYetBoundException
import static com.example.howtomocktest.ErrorLogger.handleExceptions
class ErrorLoggerSpec extends Specification {
private static final UNSUPPORTED_EXCEPTION = { throw UnsupportedOperationException }
private static final NOT_YET_BOUND = { throw NotYetBoundException }
private static final STANDARD_EXCEPTION = { throw Exception }
private Logger logger = Mock(Logger.class)
def setup() {
}
def "Message logged when UnsupportedOperationException is thrown"() {
when:
handleExceptions {UNSUPPORTED_EXCEPTION}
then:
notThrown(UnsupportedOperationException)
1 * logger.error(_ as String) // doesn't work
}
def "Message logged when NotYetBoundException is thrown"() {
when:
handleExceptions {NOT_YET_BOUND}
then:
notThrown(NotYetBoundException)
1 * logger.error(_ as String) // doesn't work
}
def "Message about processing exception is logged when standard Exception is thrown"() {
when:
handleExceptions {STANDARD_EXCEPTION}
then:
notThrown(STANDARD_EXCEPTION)
1 * logger.error(_ as String) // doesn't work
}
}
The logger in ErrorLogger class is provided by StaticLoggerBinder, so my question is - how do I make it work so that those checks "1 * logger.error(_ as String)" would work? I can't find a proper way of mocking that logger inside of ErrorLogger class. I have thought about reflection and somehow accessing it, furthermore there was an idea with mockito injection (but how to do that if reference to an object is not even present in that class because of that Slf4j annotation!) Thanks in advance for all your feedback and advices.
EDIT: Here is an output of a test, even 1*logger.error(_) doesn't work.
Too few invocations for:
1*logger.error() (0 invocations)
Unmatched invocations (ordered by similarity):
What you would need to do is to replace the log field generated by the #Slf4j AST transformation with your mock.
However, this is not so easy to achieve, since the generated code is not really test-friendly.
A quick look at the generated code reveals that it corresponds to something like this:
class ErrorLogger {
private final static transient org.slf4j.Logger log =
org.slf4j.LoggerFactory.getLogger(ErrorLogger)
}
Since the log field is declared as private final it is not so easy to replace the value with your mock. It actually boils down to the exact same problem as described here. In addition, usages of this field is wrapped in isEnabled() methods, so for instance every time you invoke log.error(msg) it is replaced with:
if (log.isErrorEnabled()) {
log.error(msg)
}
So, how to solve this? I would suggest that you register an issue at the groovy issue tracker, where you ask for a more test-friendly implementation of the AST transformation. However, this won't help you much right now.
There are a couple of work-around solutions to this that you might consider.
Set the new field value in your test using the "awful hack" described in the stack overflow question mentioned above. I.e. make the field accessible using reflection and set the value. Remember to reset the value to the original during cleanup.
Add a getLog() method to your ErrorLogger class and use that method for access instead of direct field access. Then you may manipulate the metaClass to override the getLog() implementation. The problem with this approach is that you would have to modify the production code and add a getter, which kind of defies the purpose of using #Slf4j in the first place.
I'd also like to point out that there are several problems with your ErrorLoggerSpec class. These are hidden by the problems you've already encountered, so you would probably figure these out by yourself when they manifested themselves.
Even though it is a hack, I'll only provide code example for the first suggestion, since the second suggestion modifies the production code.
To isolate the hack, enable simple reuse and avoid forgetting to reset the value, I wrote it up as a JUnit rule (which can also be used in Spock).
import org.junit.rules.ExternalResource
import org.slf4j.Logger
import java.lang.reflect.Field
import java.lang.reflect.Modifier
public class ReplaceSlf4jLogger extends ExternalResource {
Field logField
Logger logger
Logger originalLogger
ReplaceSlf4jLogger(Class logClass, Logger logger) {
logField = logClass.getDeclaredField("log");
this.logger = logger
}
#Override
protected void before() throws Throwable {
logField.accessible = true
Field modifiersField = Field.getDeclaredField("modifiers")
modifiersField.accessible = true
modifiersField.setInt(logField, logField.getModifiers() & ~Modifier.FINAL)
originalLogger = (Logger) logField.get(null)
logField.set(null, logger)
}
#Override
protected void after() {
logField.set(null, originalLogger)
}
}
And here is the spec, after fixing all the small bugs and adding this rule. Changes are commented in the code:
import org.junit.Rule
import org.slf4j.Logger
import spock.lang.Specification
import java.nio.channels.NotYetBoundException
import static ErrorLogger.handleExceptions
class ErrorLoggerSpec extends Specification {
// NOTE: These three closures are changed to actually throw new instances of the exceptions
private static final UNSUPPORTED_EXCEPTION = { throw new UnsupportedOperationException() }
private static final NOT_YET_BOUND = { throw new NotYetBoundException() }
private static final STANDARD_EXCEPTION = { throw new Exception() }
private Logger logger = Mock(Logger.class)
#Rule ReplaceSlf4jLogger replaceSlf4jLogger = new ReplaceSlf4jLogger(ErrorLogger, logger)
def "Message logged when UnsupportedOperationException is thrown"() {
when:
handleExceptions UNSUPPORTED_EXCEPTION // Changed: used to be a closure within a closure!
then:
notThrown(UnsupportedOperationException)
1 * logger.isErrorEnabled() >> true // this call is added by the AST transformation
1 * logger.error(null) // no message is specified, results in a null message: _ as String does not match null
}
def "Message logged when NotYetBoundException is thrown"() {
when:
handleExceptions NOT_YET_BOUND // Changed: used to be a closure within a closure!
then:
notThrown(NotYetBoundException)
1 * logger.isErrorEnabled() >> true // this call is added by the AST transformation
1 * logger.error(null) // no message is specified, results in a null message: _ as String does not match null
}
def "Message about processing exception is logged when standard Exception is thrown"() {
when:
handleExceptions STANDARD_EXCEPTION // Changed: used to be a closure within a closure!
then:
notThrown(Exception) // Changed: you added the closure field instead of the class here
//1 * logger.isErrorEnabled() >> true // this call is NOT added by the AST transformation -- perhaps a bug?
1 * logger.error(_ as String, _ as Exception) // in this case, both a message and the exception is specified
}
}
If you are using Spring, you have acces to OutputCaptureRule
#Rule
OutputCaptureRule outputCaptureRule = new OutputCaptureRule()
def test(){
outputCaptureRule.getAll().contains("<your test output>")
}

specs2: Is there a way to use doNothing while mocking with Mockito?

Let's say that I have a mocked trait Foo:
trait Foo {
def op(x: String): Unit
}
and I mocked this interface using
val mockedFoo = mock[Foo]
I want the method op to throw an exception second time I call it, e.g.
import org.specs2.mock.Mockito
import org.specs2.mutable.Specification
trait Foo {
def op(x: String): Unit
}
class DummySpec extends Specification with Mockito {
"dummy" should {
"test" in {
val mockedFoo = mock[Foo]
org.mockito.Mockito.doNothing().doThrow(new RuntimeException).when(mockedFoo).op(any[String])
mockedFoo.op("This one should work fine") should not(throwAn[Exception])
mockedFoo.op("This one should throw an exception") should throwAn[Exception]
}
}
}
Is there a way to do this in specs2 style? e.g.
mockedFoo.op(any[String]) returns Unit thenThrows new RuntimeException
but this doesn't compile.
Thanks!
The Unit return type makes things a bit trickier as you can't just chain:
returns "foo" thenThrows new RuntimeException
But you can still solve this problem if you use answers like below:
mockedFoo.op(anyString) answers {args => } thenThrows new RuntimeException
See if this works for you.