Make Logback throw exception on ERROR level log events - unit-testing

Question

When running unit tests, I'd like to fail any tests during which ERROR level message is logged. What would be the easiest way to achieve this using SLF4J/Logback? I'd like to avoid writing my own ILoggerFactory implementation.
I tried writing a custom Appender, but I cannot propagate exceptions through the code that's calling the Appender, all exceptions from Appender get caught there.

Answer 1

The key is to write a custom appender. You don't say which unit testing framework you use, but for JUnit I needed to do something similar (it was a little more complex than just all errors, but basically the same concept), and created a JUnit #Rule that added my appender, and the appender fails the test as needed.
I place my code for this answer in the public domain:
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.LoggerContext;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.AppenderBase;
import org.junit.rules.ExternalResource;
import org.slf4j.LoggerFactory;
import static org.junit.Assert.fail;
/**
* A JUnit {#link org.junit.Rule} which attaches itself to Logback, and fails the test if an error is logged.
* Designed for use in some tests, as if the system would log an error, that indicates that something
* went wrong, even though the error was correctly caught and logged.
*/
public class FailOnErrorLogged extends ExternalResource {
private FailOnErrorAppender appender;
#Override
protected void before() throws Throwable {
super.before();
final LoggerContext loggerContext = (LoggerContext)(LoggerFactory.getILoggerFactory());
final Logger rootLogger = (Logger)(LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME));
appender = new FailOnErrorAppender();
appender.setContext(loggerContext);
appender.start();
rootLogger.addAppender(appender);
}
#Override
protected void after() {
appender.stop();
final Logger rootLogger = (Logger)(LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME));
rootLogger.detachAppender(appender);
super.after();
}
private static class FailOnErrorAppender extends AppenderBase<ILoggingEvent> {
#Override
protected void append(final ILoggingEvent eventObject) {
if (eventObject.getLevel().isGreaterOrEqual(Level.ERROR)) {
fail("Error logged: " + eventObject.getFormattedMessage());
}
}
}
}
An example of usage:
import org.junit.Rule;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ExampleTest {
private static final Logger log = LoggerFactory.getLogger(ExampleTest.class);
#Rule
public FailOnErrorLogged failOnErrorLogged = new FailOnErrorLogged();
#Test
public void testError() {
log.error("Test Error");
}
#Test
public void testInfo() {
log.info("Test Info");
}
}
The testError method fails and the testInfo method passes. It works the same if the test calls the real class-under-test that logs an error as well.

Answer 2

Logging frameworks are generally designed not to throw any exceptions to the user. Another option (in addition to Raedwald's answer) would be to create a custom appender that sets a static boolean flag to true when an ERROR message is logged, reset this flag in a setup method and check it in a teardown method (or create a JUnit rule to reset/check the flag).

Answer 3

So, you want to fail your test case if any error reporting message of the logger is called.
Use dependency injection to associate the code to be tested with the logger it should use.
Implement a test double that implements the SLF4J logger interface, and which does nothing for most methods, but throws an AssertionError for the error logging methods.
In the set-up part of the test case, inject the test double.