I use the Boost Test framework to unit test my C++ code and wondered if it is possible to test if a function will assert? Yes, sounds a bit strange but bear with me! Many of my functions check the input parameters upon entry, asserting if they are invalid, and it would be useful to test for this. For example:
void MyFunction(int param)
{
assert(param > 0); // param cannot be less than 1
...
}
I would like to be able to do something like this:
BOOST_CHECK_ASSERT(MyFunction(0), true);
BOOST_CHECK_ASSERT(MyFunction(-1), true);
BOOST_CHECK_ASSERT(MyFunction(1), false);
...
You can check for exceptions being thrown using Boost Test so I wondered if there was some assert magic too...
Having the same problem, I digged through the documentation (and code) and
found a "solution".
The Boost UTF uses boost::execution_monitor (in
<boost/test/execution_monitor.hpp>). This is designed with the aim to catch
everything that could happen during test execution. When an assert is found
execution_monitor intercepts it and throws boost::execution_exception. Thus,
by using BOOST_REQUIRE_THROW you may assert the failure of an assert.
so:
#include <boost/test/unit_test.hpp>
#include <boost/test/execution_monitor.hpp> // for execution_exception
BOOST_AUTO_TEST_CASE(case_1)
{
BOOST_REQUIRE_THROW(function_w_failing_assert(),
boost::execution_exception);
}
Should do the trick. (It works for me.)
However (or disclaimers):
It works for me. That is, on Windows XP, MSVC 7.1, boost 1.41.0. It might
be unsuitable or broken on your setup.
It might not be the intention of the author of Boost Test.
(although it seem to be the purpose of execution_monitor).
It will treat every form of fatal error the same way. I e it could be
that something other than your assert is failing. In this case you
could miss e g a memory corruption bug, and/or miss a failed failed assert.
It might break on future boost versions.
I expect it would fail if run in Release config, since the assert will be
disabled and the code that the assert was set to prevent will
run. Resulting in very undefined behavior.
If, in Release config for msvc, some assert-like or other fatal error
would occur anyway it would not be caught. (see execution_monitor docs).
If you use assert or not is up to you. I like them.
See:
http://www.boost.org/doc/libs/1_41_0/libs/test/doc/html/execution-monitor/reference.html#boost.execution_exception
the execution-monitor user-guide.
Also, thanks to Gennadiy Rozental (Author of Boost Test), if you happen to
read this, Great Work!!
There are two kinds of errors I like to check for: invariants and run-time errors.
Invariants are things that should always be true, no matter what. For those, I use asserts. Things like you shouldn't be passing me a zero pointer for the output buffer you're giving me. That's a bug in the code, plain and simple. In a debug build, it will assert and give me a chance to correct it. In a retail build, it will cause an access violation and generate a minidump (Windows, at least in my code) or a coredump (Mac/unix). There's no catch that I can do that makes sense to deal with dereferencing a zero pointer. On Windows catch (...) can suppress access violations and give the user a false sense of confidence that things are OK when they've already gone horribly, horribly wrong.
This is one reason why I've come to believe that catch (...) is generally a code smell in C++ and the only reasonable place where I can think of that being present is in main (or WinMain) right before you generate a core dump and politely exit the app.
Run-time errors are things like "I can't write this file because of permissions" or "I can't write this file because the disk is full". For these sorts of errors throwing an exception makes sense because the user can do something about it like change the permission on a directory, delete some files or choose an alternate location to save the file. These run-time errors are correctable by the user. A violation of an invariant can't be corrected by the user, only by a programmer. (Sometimes the two are the same, but typically they aren't.)
Your unit tests should force code to throw the run-time error exceptions that your code could generate. You might also want to force exceptions from your collaborators to ensure that your system under test is exception safe.
However, I don't believe there is value in trying to force your code to assert against invariants with unit tests.
I don't think so. You could always write your own assert which throws an exception and then use BOOST_CHECK_NOTHROW() for that exception.
I think this question, and some of replies, confuse run-time errors detection with bug detection. They also confuse intent and mechanism.
Run-time error is something that can happen in a 100% correct program. It need detection, and it needs proper reporting and handling, and it should be tested. Bugs also happen, and for programmer's convenience it's better to catch them early using precondition checks or invariant checks or random assert. But this is programmer's tool. The error message will make no sense for ordinary user, and it does not seem reasonable to test function behaviour on the data that properly written program will never pass to it.
As for intent and mechanism, it should be noted that exception is nothing magic. Some time ago, Peter Dimov said on Boost mailing list (approximately) that "exceptions are just non-local jump mechanism". And this is very true. If you have application where it's possible to continue after some internal error, without the risk that something will be corrupted before repair, you can implement custom assert that throws C++ exception. But it would not change the intent, and won't make testing for asserts much more reasonable.
At work I ran into the same problem. My solution is to use a compile flag. When my flag GROKUS_TESTABLE is on my GROKUS_ASSERT is turned into an exception and with Boost you can test code paths that throw exceptions. When GROKUS_TESTABLE is off, GROKUS_ASSERT is translated to c++ assert().
#if GROKUS_TESTABLE
#define GROKUS_ASSERT ... // exception
#define GROKUS_CHECK_THROW BOOST_CHECK_THROW
#else
#define GROKUS_ASSERT ... // assert
#define GROKUS_CHECK_THROW(statement, exception) {} // no-op
#endif
My original motivation was to aid debugging, i.e. assert() can be debugged quickly and exceptions often are harder to debug in gdb. My compile flag seems to balance debuggability and testability pretty well.
Hope this helps
Sorry, but you're attacking your problem the wrong way.
"assert" is the spawn of the devil (a.k.a. "C") and is useless with any language that has proper exceptions. It's waaaaaay better to reimplement an assert-like functionality with exceptions. This way you actually get a chance of handling errors the right way (incl proper cleanup procedures) or triggering them at will (for unit testing).
Besides, if your code ever runs in Windows, when you fail an assertion you get a useless popup offering you to debug/abort/retry. Nice for automated unit tests.
So do yourself a favor and re-code an assert function that throws exceptions. There's one here:
How can I assert() without using abort()?
Wrap it in a macro so you get _ _FILE _ _ and _ _ LINE _ _ (useful for debug) and you're done.
Related
I want to unit test a C++ function which throws aborting assertion error on invalid input.
The function is as follows:
uint64_t FooBar::ReadTimeStamp(std::string& name) {
auto iter = hash_table_.find(name);
assert(iter != hash_table_.end());
....
}
In unit test, I use CPPUNIT_ASSERT_ASSERTION_FAIL to assert on assertion failure:
void FooBarTest::
TestReadNonexistentTimestamp() {
CPPUNIT_ASSERT_ASSERTION_FAIL(ReadTimestamp("NON_EXISTENT"));
}
But I got abort message and unit test failed.
I read this page. It's not clear to me if I need to throw exception here and what the correct way to unit test this scenario would be. Thanks!
Firstly, your misunderstanding is caused by the different ways to use the term "assertion". The test framework itself talks about assertions, but it does not mean the assert() macro provided by the standardlibrary. Since the standard assertion failure causes program termination, you get those results.
Now, how to fix that:
Don't use assert(). Instead, you could throw an exception.
Don't test this code path. Since this is a programming error that's not recoverable anyway, it can only be caused by misuse of your code (i.e. violating preconditions). Since it's not your code that's at fault, not testing it doesn't have any negative impact on its quality.
Hijack assert() to throw a failure that CppUnit understands. This could be tricky because for one, assert() is part of C++ and shouldn't be redefined carelessly (perhaps substituting it with a different macro would be better). Further, you now have three different behaviours: Throw in tests, abort() in regular use, UB with NDEBUG defined.
Which of these works best is up to you to decide, based on your use case and, admittedly, personal preference.
Running into an issue with some code I'm working on. This code is being run on a linux-based system and the error I receive is the following:
/root/cvswork/pci_sync_card/Code/SSBSupport/src/CRCWbHfChannel/CRCWbHfMSBSimulator.cpp:447:
virtual void CCRCWbHfMSBSimulator::Process(): Assertion 'pcBasebandOutput' failed.
I've tried stepping through this code to figure out why this is failing and I can't seem to figure it out. Unfortunately I have too many files to really share the code on here (stepping through the pcBasebandOutput assignment takes quite some time). I understand this is a more complex issue than can really be asked about. My primary questions are these:
Is my assert(pcBasebandOutput); line of code necessary? I only ask because when running this code on Visual Studio, the results from my program were desirable.
When it is evaluating my pcBasebandOutput variable, why would it evaluate it as false? Is this saying that no value is actually assigned to pcBasebandOutput? Or that a value may be assigned to it, but it is not of the right type (pointer to a struct of two variables, both of which are doubles)?
Thanks!
assert checks a logical condition. Assertation fails if the condition is false. So writing assert(cond) is logically the same as writing:
if (!cond)
{
assert(false);
}
I don't suggest you to remove assert from the code, because it is a guard telling you that something went not the way it's intended to go. And it's not a god idea just to ignore that, because it may shoot you in a leg later
Only you can know that
What is the type of pcBasebandOutput ? Maybe it is not properly initialized?
assert primary purpose is to allow your IDE to enter debuging session in the place where assert has hit. From there you can read all variables and see callstack/threads. Other solution (than using debugger) is to add lots of logging, which in threaded environments can cause problems on its own (logging is quite slow).
example of using assertions during debugging:
char* append(char* pStr, const char* pAddStr)
{
// Verify non-null pointers
assert(pStr != nullptr);
assert(pAddStr != nullptr);
// Code to append pAddStr to pStr...
}
Calling the append() function with a null pointer argument in a simple program produced the following diagnostic message on my machine:
Assertion failed: pStr != nullptr, file c:\beginning visual c++ 2010\examples visual studio project files\tryassertion\tryassertion\tryassertion.cpp, line 10
I would like to know if assertions are neccesary. What is the point of using them if i can use if-else expressions to output my own error messages?
An assertion is a conditional check; it's just a macro that's something like this (simplified):
#define assert(b) if (!(b)) { std::cerr << ...; exit(1); }
I can think of two advantages:
When you compile in release mode (i.e. not in debug mode), then all the asserts will compile away to nothing, so you will incur no runtime overhead.
assert is an idiom; other programmers know to look for them, and they're also distinct from "normal" control flow.
assertions are used to ensure that certain basic assumptions are met. Basically you put in an assertation in every case which "couldnt possibly happen", most commonly assertions how the API has to be used e.g. preconditions and postconditions (like your example, which contains asserts which check the correct usage of your append function). For other errors, which are known to occur during runtime and which cannot be prevented beforehand (e.g. file-not-found or insufficient-permissions kind of errors), you will have to write error handling code.
Assertions are not included in release compilations, so they can only be used to catch critical programming errors, which will already occur in debug-mode test runs.
You should use assertions when the only case in which they are violated is an error in the program logic. You use normal if-then-else conditionals for things that may indeed happen because of input or external possible conditions (i.e. a file is missing).
When you know for sure that the program logic is faulty there is not really much sense in trying to continue execution... your program is working differently from what you think (because otherwise the assertion wouldn't have been triggered) and therefore the best option is just to yell what the problem is and die immediately.
Often assertion are removed when the code is compiled in "release" mode even if however it may make sense to keep them in place if the program logic is very complex and if continuing execution generating bad output is going to create bigger problems than stopping execution.
Note that a "trap" that sometimes novice programmers fall into with assertion is that when the assertion code is removed for release mode the expression inside the assert is not evaluated any more, and therefore if you program depends on side effects of that expression then you're going to be in trouble... for example:
...
assert(insert_record(db, data) == DB_OK); // <== this is bad
...
when assertion are defined away the insertion will not happen at all, leaving you with a program that doesn't work in release mode and that works instead when you try to debug the problem.
I am just learning how to handle errors in my C++ code. I wrote this example that looks for a text file called some file, and if its not found will throw an exception.
#include <iostream>
#include <fstream>
using namespace std;
int main()
{
int array[90];
try
{
ifstream file;
file.open("somefile.txt");
if(!file.good())
throw 56;
}
catch(int e)
{
cout<<"Error number "<<e<<endl;
}
return 0;
}
Now I have two questions. First I would like to know if I am using Exceptions correctly. Second, (assuming the first is true) what is the benefit to using them vs an If else statement?
"Correctly" is a value judgment, but (unlike other classes) there's a major benefit from exceptions classes being a monolithic hierarchy, so I'd generally advise throwing something derived from std::exception, not simply an int.
Second, it's open to question whether an incorrect file name is sufficiently unexpected to qualify as a good reason to throw an exception at all.
As to benefits vs. an if/else statement: there are a couple. First, exceptions let you segregate the code that deals with errors, so the main idea and readability of the code don't get lost in a maze of error handling. Second, when you have several layers of code between throwing and catching the exception, the code that throws the exception may not know how it should really be handled. Your code, for example, uses std::cout to report the problem -- but most such code would report errors on std::cerr instead. You can change from one to the other without any change to the code that tried to open the file (which might be deep in a library, and have no clue of which should be used for this application -- and might be used in an application where both are wrong, and MessageBox was preferred).
First I would like to know if I am using Exceptions correctly.
Yes, though generally you want your exceptions to derive from std::exception.
Second, (assuming the first is true) what is the benefit to using them vs an If else statement?
For the given example, nothing. The benefit of exceptions comes when you have
many deep nested functions, like this.
#include <stdexcept>
#include <iostream>
#include <string>
void anErrorFunc(const std::string& x)
{
ifstream file;
file.open(x);
if (!file)
throw std::runtime_error("Could not open file");
}
void someOtherFunction(const std::string& y)
{
//Do stuff
anErrorFunc(y);
//Do other stuff
}
int main()
{
try {
someOtherFunction("somefile.txt");
} catch (std::exception &ex) {
std::cout << "Ouch! That hurts, because: "
<< ex.what() << "!\n";
}
}
Note that the exception will be caught in main(), and someOtherFunction
does not have to worry about dealing with passing through failure return codes.
Well, you are using exception correctly in that there is nothing wrong with your code. That said, typically we do not throw primitive types around (even though you can). It is generally a better idea to throw an object that derives from std::exception, and even better to throw an std::exception that is also a boost::exception.
When things are very simple and the handling code and the throwing code are in the same function, then there really is no reason to use exceptions instead of if statements (and, indeed, it would be faster and more efficient to use if...else in that particular case). However, in most situations, the point where the error is discovered and you need to report it is far removed from the logic where the error is to be handled. In many cases the error-recovery logic is specific to the application in question and the logic where the error is discovered cannot make a sensible choice about how to recover from the error, hence the need to throw.
Another benefit of exception handling is that the type of the exception can be used to convey the type of error that occurred. Usually the types in exception hierarchies are much more meaningful than those error codes that end up being used in C code. Also, you cannot ignore an exception as easily as you can ignore an error code; while you can ignore an exception, it will cause the program to die a horrible death. By contrast, if a C function returns an error status code, and you ignore it, it's possible to continue executing and get silently wrong results... in that sense, the use of exceptions is much safer than using error codes.
You may also be interested in reading about exceptions and error handling from the C++ FAQ Lite.
You are not using exceptions correctly. Your code has a much simpler equivalent, without exceptions, which provides the same behaviour.
Exceptions are for when you can't test the results of a function call with other methods. In this case you can, so you shouldn't use exceptions.
As a corollary, you shouldn't throw and catch the exception in the same function body - just do whatever you want to do instead of throwing it.
Syntactically speaking, your code is correct. Idiomatically speaking, maybe not so much -- or at least this depends on the context. When a file can't open then we might do our handling right there inside that if( !file.good() ) if it's perfectly common and possible to happen. For example, if the user asks to open a file in a text editor then it's perfectly plausible and common that the file doesn't exist. On the other hand, if the editor can't find the spelling corpus file then that means something is (arguably) terribly wrong. The program wasn't probably installed, or the user messed around with that file -- anything is possible.
In C++ we use exceptions for exceptional cases. That is, cases that are really not meant to happen and that we don't "accept" to happen. This is as opposed to a user file not opening, or an invalid user input, or no internet connection: these are all examples of perfectly valid things happening, common situations, things we expect to happen sooner or later in a program's run. They aren't exceptional.
Now, what are the benefits of using exceptions compared to conditionals? Allow me to extend this question to any other jump (goto) mechanism: returns as well as conditionals. Exceptions are more expressive if that's what you want to say: if you are dealing with an exceptional case then use exceptions. Exceptions also get more done than plain conditionals, in an analogous way to that of virtual functions getting more done than conditionals. The right code-block will be executed depending on the exception, but also the right scope will handle the exception depending on the handlers.
There are other advantages to exceptions compared with conditionals: exceptions separate error-handling from other code. They allow associating arbitrary data and actions with an error state. They allow communicating success states (via a return) as well as error state (via a throw). And the list goes on...
Technically speaking and at the lowest level exceptions are a sophisticated jump mechanism. Back in the butterfly-days people invented the if conditional as a somewhat-sophisticated goto in order to enhance expressiveness (because a goto can be used for anything really) and to reduce programmer errors. The looping constructs such as the C for loop are also in essence a sophisticated jump with sparkles and rainbow colours, again for reducing errors and enhancing expressiveness. C++ introduced its new casting operators for the same reasons.
So there: exceptions aren't magic, just something somewhat new in the scene compared to conditionals and loops. Don't use them when you don't mean to, just like you don't use a loop when you really mean to use a conditional.
Syntactically what you're doing is right. Style-wise, as others have noted, you should be throwing something descended from std::exception.
As to part two of your question, I'd like to go into more detail on that.
The whole point of exceptions is to separate policy from implementation. As Billy ONeal said, you get no benefit at all from using exceptions within the same function that an if statement wouldn't make better. You need to be deeply nested in function calls for it to make sense.
In most code, your high level code has enough information and context to know what to do about errors, but no mechanism to detect them. Your low level code can detect the errors but has none of the information needed to deal with them.
The traditional means of coping with this -- returning error codes -- has a few problems:
It clutters up the code with error handling code to the point that the actual logic is obfuscated.
It relies on programmers not being lazy and checking EVERY error code return, an often foolhardy assumption. (C programmers, be honest here: when was the last time you checked the return value of printf?)
It adds the overhead of error checking and handling to EVERY function call whether there's an error or not.
Exceptions solve these issues (with varying degrees of success).
Exceptions solve #1 by only having exception-related code at the point of detection and at the point of handling. Intervening functions don't get cluttered with handling for obscure errors that they themselves have no interest in nor capability of dealing with.
They solve #2 by forcing handling. You can't ignore an exception. You have to take action on them. (Lazy programmers can still catch all exceptions and then ignore them, but here their crippling inability to program is now highlighted for all to see.)
They solve #3 (when not naively implemented) by having near-zero costs when not used, albeit often at a very, very high cost when actually used.
This is not to say that exceptions are the end-all/be-all of error handling. The disadvantages:
Exceptions are usually very costly when used. They have to be eschewed, despite their advantages, if performance is paramount.
Exceptions lead to very opaque code at times. They are non-local transfers of control -- effectively slightly safer versions of goto statements, but across functions. An exception can transfer control from hundreds of layers deep in your code in source files not even slightly related to the ones you're working on (and, in fact, quite possibly not even accessible to you). This kind of "spooky action at a distance" can make code very difficult to figure out.
"Checked exceptions" can actually be worse for noise generation than the old-style if handling. They tend to be more verbose, you see, than if or switch statements and the fact that you must handle checked exceptions for the code to even compile makes them a liability to many situations.
Because of their often high cost of use, carelessly using them for all error handling can make your code slow and bloated.
How are assertions done in c++? Example code is appreciated.
Asserts are a way of explicitly checking the assumptions that your code makes, which helps you track down lots of bugs by narrowing down what the possible problems could be. They are typically only evaluated in a special "debug" build of your application, so they won't slow down the final release version.
Let's say you wrote a function that took a pointer as an argument. There's a good chance that your code will assume that the pointer is non-NULL, so why not explicitly check that with an assertion? Here's how:
#include <assert.h>
void function(int* pointer_arg)
{
assert(pointer_arg != NULL);
...
}
An important thing to note is that the expressions you assert must never have side effects, since they won't be present in the release build. So never do something like this:
assert(a++ == 5);
Some people also like to add little messages into their assertions to help give them meaning. Since a string always evaulates to true, you could write this:
assert((a == 5) && "a has the wrong value!!");
Assertion are boolean expressions which should typically always be true.
They are used to ensure what you expected is also what happens.
void some_function(int age)
{
assert(age > 0);
}
You wrote the function to deal with ages, you also 'know' for sure you're always passing sensible arguments, then you use an assert. It's like saying "I know this can never go wrong, but if it does, I want to know", because, well, everyone makes mistakes.
So it's not to check for sensible user input, if there are scenario's where something can go wrong, don't use an assert. Do real checks and deal with the errors.
Asserts are typically only for debug builds, so don't put code with side effects in asserts.
Assertions are used to verify design assumptions, usually in terms of input parameters and return results. For example
// Given customer and product details for a sale, generate an invoice
Invoice ProcessOrder(Customer Cust,Product Prod)
{
assert(IsValid(Cust));
assert(IsValid(Prod);
'
'
'
assert(IsValid(RetInvoice))
return(RetInvoice);
}
The assert statements aren't required for the code to run, but they check the validity of the input and output. If the input is invalid, there is a bug in the calling function. If the input is valid and output is invalid, there is a bug in this code. See design by contract for more details of this use of asserts.
Edit: As pointed out in other posts, the default implementation of assert is not included in the release run-time. A common practice that many would use, including myself, is to replace it with a version that is included in the release build, but is only called in a diagnostics mode. This enables proper regression testing on release builds with full assertion checking. My version is as follows;
extern void _my_assert(void *, void *, unsigned);
#define myassert(exp) \
{ \
if (InDiagnostics) \
if ( !(exp) ) \
_my_assert(#exp, __FILE__, __LINE__); \
} \
There is a small runtime overhead in this technique, but it makes tracking any bugs that have made it into the field much easier.
Use assertions to check for "can't happen" situations.
Typical usage: check against invalid/impossible arguments at the top of a function.
Seldom seen, but still useful: loop invariants and postconditions.
Assertions are statements allowing you to test any assumptions you might have in your program. This is especially useful to document your program logic (preconditions and postconditions). Assertions that fail usually throw runtime errors, and are signs that something is VERY wrong with your program - your assertion failed because something you assumed to be true was not. The usual reasons are: there is a flaw in your function's logic, or the caller of your function passed you bad data.
An assertion is something you add to your program that causes the program to stop immediately if a condition is met, and display an error message. You generally use them for things which you believe can never happen in your code.
This doesn't address the assert facility which has come down to us from early C days, but you should also be aware of Boost StaticAssert functionality, in the event that your projects can use Boost.
The standard C/C++ assert works during runtime. The Boost StaticAssert facility enables you to make some classes of assertions at compile time, catching logic errors and the like even earlier.
Here is a definition of what an assertion is and here is some sample code. In a nutshell an assertion is a way for a developer to test his (or her) assumptions about the state of the code at any given point. For example, if you were doing the following code:
mypointer->myfunct();
You probably want to assert that mypointer is not NULL because that's your assumption--that mypointer will never be NULL before the call.