Testing assertion in Swift - unit-testing

Question

I'm writing unit tests for a method that has an assertion. The Swift Language guide recommends using assertions for "invalid conditions":
Assertions cause your app to terminate and are not a substitute for
designing your code in such a way that invalid conditions are unlikely
to arise. Nonetheless, in situations where invalid conditions are
possible, an assertion is an effective way to ensure that such
conditions are highlighted and noticed during development, before your
app is published.
I want to test the failure case.
However, there is not XCTAssertThrows in Swift (as of Beta 6). How can I write an unit test that tests that an assertion fails?
Edit
As per #RobNapier's suggestion, I tried wrapping XCTAssertThrows in an Objective-C method and calling this method from Swift. This doesn't work as the macro does not catch the fatal error caused by assert, and thus the test crashes.

Answer 1

assert and its sibling precondition don't throw exceptions cannot be "caught" (even with Swift 2's error handling).
A trick you can use is to write your own drop-in replacement that does the same thing but can be replaced for tests. (If you're worried about performance, just #ifdef it away for release builds.)
custom precondition
/// Our custom drop-in replacement `precondition`.
///
/// This will call Swift's `precondition` by default (and terminate the program).
/// But it can be changed at runtime to be tested instead of terminating.
func precondition(#autoclosure condition: () -> Bool, #autoclosure _ message: () -> String = "", file: StaticString = __FILE__, line: UWord = __LINE__) {
preconditionClosure(condition(), message(), file, line)
}
/// The actual function called by our custom `precondition`.
var preconditionClosure: (Bool, String, StaticString, UWord) -> () = defaultPreconditionClosure
let defaultPreconditionClosure = {Swift.precondition($0, $1, file: $2, line: $3)}
test helper
import XCTest
extension XCTestCase {
func expectingPreconditionFailure(expectedMessage: String, #noescape block: () -> ()) {
let expectation = expectationWithDescription("failing precondition")
// Overwrite `precondition` with something that doesn't terminate but verifies it happened.
preconditionClosure = {
(condition, message, file, line) in
if !condition {
expectation.fulfill()
XCTAssertEqual(message, expectedMessage, "precondition message didn't match", file: file.stringValue, line: line)
}
}
// Call code.
block();
// Verify precondition "failed".
waitForExpectationsWithTimeout(0.0, handler: nil)
// Reset precondition.
preconditionClosure = defaultPreconditionClosure
}
}
example
func doSomething() {
precondition(false, "just not true")
}
class TestCase: XCTestCase {
func testExpectPreconditionFailure() {
expectingPreconditionFailure("just not true") {
doSomething();
}
}
}
(gist)
Similar code will work for assert, of course. However, since you're testing the behavior, you obviously want it to be part of your interface contract. You don't want optimized code to violate it, and assert will be optimized away. So better use precondition here.

Answer 2

Agree with nschum's comment that it doesn't seem right to unit test assert because by default it wont be in the prod code. But if you really wanted to do it, here is the assert version for reference:
override assert
func assert(#autoclosure condition: () -> Bool, #autoclosure _ message: () -> String = "", file: StaticString = __FILE__, line: UInt = __LINE__) {
assertClosure(condition(), message(), file, line)
}
var assertClosure: (Bool, String, StaticString, UInt) -> () = defaultAssertClosure
let defaultAssertClosure = {Swift.assert($0, $1, file: $2, line: $3)}
helper extension
extension XCTestCase {
func expectAssertFail(expectedMessage: String, testcase: () -> Void) {
// arrange
var wasCalled = false
var assertionCondition: Bool? = nil
var assertionMessage: String? = nil
assertClosure = { condition, message, _, _ in
assertionCondition = condition
assertionMessage = message
wasCalled = true
}
// act
testcase()
// assert
XCTAssertTrue(wasCalled, "assert() was never called")
XCTAssertFalse(assertionCondition!, "Expected false to be passed to the assert")
XCTAssertEqual(assertionMessage, expectedMessage)
// clean up
assertClosure = defaultAssertClosure
}
}

Answer 3

Thanks to nschum and Ken Ko for the idea behind this answer.
Here is a gist for how to do it
Here is an example project
This answer is not just for assert. It's also for the other assertion methods (assert, assertionFailure, precondition, preconditionFailure and fatalError)
1. Drop ProgrammerAssertions.swift to the target of your app or framework under test. Just besides your source code.
ProgrammerAssertions.swift
import Foundation
/// drop-in replacements
public func assert(#autoclosure condition: () -> Bool, #autoclosure _ message: () -> String = "", file: StaticString = __FILE__, line: UInt = __LINE__) {
Assertions.assertClosure(condition(), message(), file, line)
}
public func assertionFailure(#autoclosure message: () -> String = "", file: StaticString = __FILE__, line: UInt = __LINE__) {
Assertions.assertionFailureClosure(message(), file, line)
}
public func precondition(#autoclosure condition: () -> Bool, #autoclosure _ message: () -> String = "", file: StaticString = __FILE__, line: UInt = __LINE__) {
Assertions.preconditionClosure(condition(), message(), file, line)
}
#noreturn public func preconditionFailure(#autoclosure message: () -> String = "", file: StaticString = __FILE__, line: UInt = __LINE__) {
Assertions.preconditionFailureClosure(message(), file, line)
runForever()
}
#noreturn public func fatalError(#autoclosure message: () -> String = "", file: StaticString = __FILE__, line: UInt = __LINE__) {
Assertions.fatalErrorClosure(message(), file, line)
runForever()
}
/// Stores custom assertions closures, by default it points to Swift functions. But test target can override them.
public class Assertions {
public static var assertClosure = swiftAssertClosure
public static var assertionFailureClosure = swiftAssertionFailureClosure
public static var preconditionClosure = swiftPreconditionClosure
public static var preconditionFailureClosure = swiftPreconditionFailureClosure
public static var fatalErrorClosure = swiftFatalErrorClosure
public static let swiftAssertClosure = { Swift.assert($0, $1, file: $2, line: $3) }
public static let swiftAssertionFailureClosure = { Swift.assertionFailure($0, file: $1, line: $2) }
public static let swiftPreconditionClosure = { Swift.precondition($0, $1, file: $2, line: $3) }
public static let swiftPreconditionFailureClosure = { Swift.preconditionFailure($0, file: $1, line: $2) }
public static let swiftFatalErrorClosure = { Swift.fatalError($0, file: $1, line: $2) }
}
/// This is a `noreturn` function that runs forever and doesn't return.
/// Used by assertions with `#noreturn`.
#noreturn private func runForever() {
repeat {
NSRunLoop.currentRunLoop().run()
} while (true)
}
2. Drop XCTestCase+ProgrammerAssertions.swift to your test target. Just besides your test cases.
XCTestCase+ProgrammerAssertions.swift
import Foundation
import XCTest
#testable import Assertions
private let noReturnFailureWaitTime = 0.1
public extension XCTestCase {
/**
Expects an `assert` to be called with a false condition.
If `assert` not called or the assert's condition is true, the test case will fail.
- parameter expectedMessage: The expected message to be asserted to the one passed to the `assert`. If nil, then ignored.
- parameter file: The file name that called the method.
- parameter line: The line number that called the method.
- parameter testCase: The test case to be executed that expected to fire the assertion method.
*/
public func expectAssert(
expectedMessage: String? = nil,
file: StaticString = __FILE__,
line: UInt = __LINE__,
testCase: () -> Void
) {
expectAssertionReturnFunction("assert", file: file, line: line, function: { (caller) -> () in
Assertions.assertClosure = { condition, message, _, _ in
caller(condition, message)
}
}, expectedMessage: expectedMessage, testCase: testCase) { () -> () in
Assertions.assertClosure = Assertions.swiftAssertClosure
}
}
/**
Expects an `assertionFailure` to be called.
If `assertionFailure` not called, the test case will fail.
- parameter expectedMessage: The expected message to be asserted to the one passed to the `assertionFailure`. If nil, then ignored.
- parameter file: The file name that called the method.
- parameter line: The line number that called the method.
- parameter testCase: The test case to be executed that expected to fire the assertion method.
*/
public func expectAssertionFailure(
expectedMessage: String? = nil,
file: StaticString = __FILE__,
line: UInt = __LINE__,
testCase: () -> Void
) {
expectAssertionReturnFunction("assertionFailure", file: file, line: line, function: { (caller) -> () in
Assertions.assertionFailureClosure = { message, _, _ in
caller(false, message)
}
}, expectedMessage: expectedMessage, testCase: testCase) { () -> () in
Assertions.assertionFailureClosure = Assertions.swiftAssertionFailureClosure
}
}
/**
Expects an `precondition` to be called with a false condition.
If `precondition` not called or the precondition's condition is true, the test case will fail.
- parameter expectedMessage: The expected message to be asserted to the one passed to the `precondition`. If nil, then ignored.
- parameter file: The file name that called the method.
- parameter line: The line number that called the method.
- parameter testCase: The test case to be executed that expected to fire the assertion method.
*/
public func expectPrecondition(
expectedMessage: String? = nil,
file: StaticString = __FILE__,
line: UInt = __LINE__,
testCase: () -> Void
) {
expectAssertionReturnFunction("precondition", file: file, line: line, function: { (caller) -> () in
Assertions.preconditionClosure = { condition, message, _, _ in
caller(condition, message)
}
}, expectedMessage: expectedMessage, testCase: testCase) { () -> () in
Assertions.preconditionClosure = Assertions.swiftPreconditionClosure
}
}
/**
Expects an `preconditionFailure` to be called.
If `preconditionFailure` not called, the test case will fail.
- parameter expectedMessage: The expected message to be asserted to the one passed to the `preconditionFailure`. If nil, then ignored.
- parameter file: The file name that called the method.
- parameter line: The line number that called the method.
- parameter testCase: The test case to be executed that expected to fire the assertion method.
*/
public func expectPreconditionFailure(
expectedMessage: String? = nil,
file: StaticString = __FILE__,
line: UInt = __LINE__,
testCase: () -> Void
) {
expectAssertionNoReturnFunction("preconditionFailure", file: file, line: line, function: { (caller) -> () in
Assertions.preconditionFailureClosure = { message, _, _ in
caller(message)
}
}, expectedMessage: expectedMessage, testCase: testCase) { () -> () in
Assertions.preconditionFailureClosure = Assertions.swiftPreconditionFailureClosure
}
}
/**
Expects an `fatalError` to be called.
If `fatalError` not called, the test case will fail.
- parameter expectedMessage: The expected message to be asserted to the one passed to the `fatalError`. If nil, then ignored.
- parameter file: The file name that called the method.
- parameter line: The line number that called the method.
- parameter testCase: The test case to be executed that expected to fire the assertion method.
*/
public func expectFatalError(
expectedMessage: String? = nil,
file: StaticString = __FILE__,
line: UInt = __LINE__,
testCase: () -> Void) {
expectAssertionNoReturnFunction("fatalError", file: file, line: line, function: { (caller) -> () in
Assertions.fatalErrorClosure = { message, _, _ in
caller(message)
}
}, expectedMessage: expectedMessage, testCase: testCase) { () -> () in
Assertions.fatalErrorClosure = Assertions.swiftFatalErrorClosure
}
}
// MARK:- Private Methods
private func expectAssertionReturnFunction(
functionName: String,
file: StaticString,
line: UInt,
function: (caller: (Bool, String) -> Void) -> Void,
expectedMessage: String? = nil,
testCase: () -> Void,
cleanUp: () -> ()
) {
let expectation = expectationWithDescription(functionName + "-Expectation")
var assertion: (condition: Bool, message: String)? = nil
function { (condition, message) -> Void in
assertion = (condition, message)
expectation.fulfill()
}
// perform on the same thread since it will return
testCase()
waitForExpectationsWithTimeout(0) { _ in
defer {
// clean up
cleanUp()
}
guard let assertion = assertion else {
XCTFail(functionName + " is expected to be called.", file: file.stringValue, line: line)
return
}
XCTAssertFalse(assertion.condition, functionName + " condition expected to be false", file: file.stringValue, line: line)
if let expectedMessage = expectedMessage {
// assert only if not nil
XCTAssertEqual(assertion.message, expectedMessage, functionName + " called with incorrect message.", file: file.stringValue, line: line)
}
}
}
private func expectAssertionNoReturnFunction(
functionName: String,
file: StaticString,
line: UInt,
function: (caller: (String) -> Void) -> Void,
expectedMessage: String? = nil,
testCase: () -> Void,
cleanUp: () -> ()
) {
let expectation = expectationWithDescription(functionName + "-Expectation")
var assertionMessage: String? = nil
function { (message) -> Void in
assertionMessage = message
expectation.fulfill()
}
// act, perform on separate thead because a call to function runs forever
dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0), testCase)
waitForExpectationsWithTimeout(noReturnFailureWaitTime) { _ in
defer {
// clean up
cleanUp()
}
guard let assertionMessage = assertionMessage else {
XCTFail(functionName + " is expected to be called.", file: file.stringValue, line: line)
return
}
if let expectedMessage = expectedMessage {
// assert only if not nil
XCTAssertEqual(assertionMessage, expectedMessage, functionName + " called with incorrect message.", file: file.stringValue, line: line)
}
}
}
}
3. Use assert, assertionFailure, precondition, preconditionFailure and fatalError normally as you always do.
For example: If you have a function that does a division like the following:
func divideFatalError(x: Float, by y: Float) -> Float {
guard y != 0 else {
fatalError("Zero division")
}
return x / y
}
4. Unit test them with the new methods expectAssert, expectAssertionFailure, expectPrecondition, expectPreconditionFailure and expectFatalError.
You can test the 0 division with the following code.
func testFatalCorrectMessage() {
expectFatalError("Zero division") {
divideFatalError(1, by: 0)
}
}
Or if you don't want to test the message, you simply do.
func testFatalErrorNoMessage() {
expectFatalError() {
divideFatalError(1, by: 0)
}
}

Answer 4

Matt Gallagher's CwlPreconditionTesting project on github adds a catchBadInstruction function which gives you the ability to test for assertion/precondition failures in unit test code.
The CwlCatchBadInstructionTests file shows a simple illustration of its use. (Note that it only works in the simulator for iOS.)

Answer 5

I believe as of Beta6 it is still impossible for Swift to catch an exception directly. The only way you can handle this is to write that particular test case in ObjC.
That said, note that _XCTAssertionType.Throws does exist, which suggests that the Swift team is aware of this and intends eventually to provide a solution. It is quite imaginable that you could write this assertion yourself in ObjC and expose it to Swift (I can't think of any reason that would be impossible in Beta6). The one big problem is that you may not easily be able to get good location information out of it (the specific line that failed, for instance).

Answer 6

We have Swift (4) code that tests an Objective-C framework. Some of the framework methods call into NSAssert.
Inspired by NSHipster, I ended up with an implementation like such:
SwiftAssertionHandler.h (use this in a bridging header)
#interface SwiftAssertionHandler : NSAssertionHandler
#property (nonatomic, copy, nullable) void (^handler)(void);
#end
SwiftAssertionHandler.m
#implementation SwiftAssertionHandler
- (instancetype)init {
if (self = [super init]) {
[[[NSThread currentThread] threadDictionary] setValue:self
forKey:NSAssertionHandlerKey];
}
return self;
}
- (void)dealloc {
[[[NSThread currentThread] threadDictionary] removeObjectForKey:NSAssertionHandlerKey];
}
- (void)handleFailureInMethod:(SEL)selector object:(id)object file:(NSString *)fileName lineNumber:(NSInteger)line description:(NSString *)format, ... {
if (self.handler) {
self.handler();
}
}
- (void)handleFailureInFunction:(NSString *)functionName file:(NSString *)fileName lineNumber:(NSInteger)line description:(NSString *)format, ... {
if (self.handler) {
self.handler();
}
}
#end
Test.swift
let assertionHandler = SwiftAssertionHandler()
assertionHandler.handler = { () -> () in
// i.e. count number of assert
}