assert()s, optimisation and an assume() directive in D - d

Say I have an assert() something like
assert( x < limit );
I took a look at the behaviour of the optimiser in GDC in release and debug builds with the following snippet of code:
uint cxx1( uint x )
{
assert( x < 10 );
return x % 10;
}
uint cxx1a( uint x )
in { assert( x < 10 ); }
body
{
return x % 10;
}
uint cxx2( uint x )
{
if ( !( x < 10 ))
assert(0);
return x % 10;
}
Now when I build in debug mode, the asserts have the very pleasing effect of triggering huge optimisation. GDC gets rid of the horrid code to do the modulo operation entirely, because of its knowledge about the possible range of x due to the assert’s if-condition. But in release mode, the if-condition is discarded, so all of a sudden, the horrid code comes back, and there is no longer any optimisation in cxx1() nor even in cxx1a(). This is very ironic, that release mode generates far worse code than debug code. Of course, no-one wants executable code belonging to the if-tests to be present in release code as we must lose all that overhead.
Now ideally, I would want to express the condition in the sense of communicating information to the compiler, regardless of release / debug builds, about conditions that may always be assumed to be true, and so such assumptions can guide optimisation in very powerful ways.
I believe some C++ compilers have something called __assume() or some such, but memory fails me here. GCC has a __builtin_unreachable() special directive which might be useable to build an assume() feature. Basically if I could build my own assume() directive it would have the effect of asserting certain truths about known values or known ranges and exposing / publishing these to optimisation passes regardless of release / debug mode but without generating any actual code at all for the assume() condition in a release build, while in debug mode it would be exactly the same as assert().
I tried an experiment which you see in cxx2 which triggers optimisation always, so good job there, but it generates what is morally debug code for the assume()'s if-condition even in release mode with a test and a conditional jump to an undefined instruction in order to halt the process.
Does anyone have any ideas about whether this is solvable? Or do you think this is a useful D compiler fantasy wish-list item?

As far as I know __builtin_unreachable is the next best replacement for an assume like function in GCC. In some cases the if condition might still not get optimized out though: "Assume" clause in gcc
The GCC builtins are available in GDC by importing gcc.builtins. Here's an example how to wrap the __builtin_unreachable function:
import gcc.builtins;
void assume()(bool condition)
{
if (!condition)
__builtin_unreachable();
}
bool foo(int a)
{
assume(a > 10);
return a > 10;
}
There are two interesting details here:
We don't need string mixins or similarily complicated stuff. As long as you compile with -O GDC will completely optimize the function call anyway.
For this to work the assume function must get inlined. Unfortunately inlining normal functions is not completely supported when assume is in a different module as the calling function. As a workaround we use a template with 0 template arguments. This should make sure inlining can always work.
You can test and modify this example here:
explore.dgnu.org
Now we (GDC developers) could easily rewrite assert(...) to if(...) __builtin_unreachable() in release mode. But this could break some code so dmd should implement this first.

OK, I really dont know what you want? cxx2 is solution
some more info

Related

What are the advantages or implications of using assert vs exit or vice versa?

As far as I understand, a call to assert(e), where e is a boolean expression, executes something roughly like if (!e) { printf("%s:%d: failed assertion `%s'\n", __FILE__, __LINE__, e); abort(); } which terminates the program abruptly if the given expression is not true.
On the other hand, instead of using assert, I could probably write something like if (!e) { fprintf(stderr, "custom error message.\n"); exit(1); } which naively feels like a more cleaner, nicer thing to do.
Ignoring that assert can be turned off globally using the NDEBUG flag, what other advantage do you think one has over the other? Have I got the distinction right, or are there conceptual differences between the two which I am unaware of that let the two ways of conditional program termination have their own niche use cases? If the latter, please explain. Thank you.
The biggest advantage of assert is that it makes one's intentions clear. If you see assert(some_condition) then you know what the author's intent was (i.e., some_condition is always true). With your inlined version, I can't assume intention until I actually read your if block, and realize you're going to display an error message and terminate the program.
Less important reasons include that assert reduces copy/paste errors, some_condition is turned into a string automatically (including preservation of variable names), and that tooling can understand it.
what other advantage do you think one has over the other?
A macro is used because you want to be able to remove it via conditional compilation. In other words, you don't want the code to even appear in the binary.
Have I got the distinction right, or are there conceptual differences between the two which I am unaware of that let the two ways of conditional program termination have their own niche use cases?
Well, exit() and abort() don't behave the same way even if you use 1 as a "unsuccessful" exit code. The latter is intended to kill the program right away without further work and possibly trigger a debugging prompt or save an image of the process space (although exactly what it does depends on the vendor providing it). The former calls the registered functions by atexit(). There are other ways of stopping, too, see quick_exit() and _Exit().
For C++, there are way more considerations on the behavioral difference, e.g. whether destructors for variables in the stack frame(s) are run, whether global destructors are run, what happens if an exception is thrown while doing that, etc.
'assert' is a code autotesting tool. Sometimes the program should not stop its work with on client side (release version) even if the condition leading to the execution of the assert was met.
For example:
switch(color)
{
case red:
//...
break;
case green:
//...
break;
default:
assert(false && "unexpected color value. 'enum color' was modified?");
}
another example:
if ( OkOrFailEnum::OK != result )
{
assert(false && "data access fail");
throw std::runtime_error("Can not get index " + std::to_string(index));
}
At the same time, 'assert' is a code commenting tool.
inline unsigned Sum(unsigned* pPos, unsigned* pOffset)
{
assert(nullptr != pPos); // \
assert(nullptr != pOffset); // | preconditions
assert(*pPos + *pOffset < *pOffset && "Overflow?"); // /
return *pPos + *pOffset;
}
The assert:
assert(*pPos + *pOffset < *pOffset && "Overflow?");
means, that the Sum(..) function does not works correctly with big sums and it have to do some check before call the function.

Are comparison between macro values bad in embedded programming?

I am building a program that needs to run on an ARM.
The processor has plenty of resources to run the program, so this question is not directly related to this type of processor, but is related to non powerful ones, where resources and computing power are 'limited'.
To print debug informations (or even to activate portions of code) I am using a header file where I define macros that I set to true or false, like this:
#define DEBUG_ADCS_OBC true
and in the main program:
if (DEBUG_ADCS_OBC == true) {
printf("O2A ");
for (j = 0; j < 50; j++) {
printf("%x ", buffer_obc[jj]);
}
}
Is this a bad habit? Are there better ways to do this?
In addition, will having these IF checks affect performances in a measurable way?
Or is it safe to assume that when the code is compiled the IFs are somehow removed from the flow, as the comparison is made between two values that cannot change?
Since the expression DEBUG_ADCS_OBC == true can be evaluated at compile time, optimizing compilers will figure out that the branch is either always taken or is always bypassed, and eliminate the condition altogether. Therefore, there is zero runtime cost to the expression when you use an optimized compiler.
If you are compiling with all optimization turned off, use conditional compilation instead. This will do the same thing an optimizing compiler does with a constant expression, but at the preprocessor stage. Hence the compiler will not "see" the conditional even with optimization turned off.
Note 1: Since DEBUG_ADCS_OBC has a meaning of boolean variable, use DEBUG_ADCS_OBC without == true for a somewhat cleaner look.
Note 2: Rather than defining the value in the body of your program, consider passing a value on the command line, for example -DDEBUG_ADCS_OBC=true. This lets you change the debug setting without modifying your source code, simply by manipulating the make file or one of its options.
The code you are using is evaluated everytime when your program reaches this line. Since every change of DEBUG_ADCS_OBC will require a recompile of your code, you should use #ifdef/#ifndef expressions instead. The advantage of them is, that they are only evaluated once at compile time.
Your code segment could look like the following:
Header:
//Remove this line if debugging should be disabled
#define DEBUG_DCS_OBS
Source:
#ifdef DEBUG_DCS_OBS
printf("O2A ");
for (j = 0; j < 50; j++) {
printf("%x ", buffer_obc[jj]);
}
#endif
The problem with getting the compiler to do this is the unnecessary run-time test of a constant expression. An optimising compiler will remove this, but equally it may issue warnings about constant expressions or when the macro is undefined, issue warnings about unreachable code.
It is not a matter of "bad in embedded programming", it bears little merit in any programming domain.
The following is the more usual idiom, will not include unreachable code in the final build and in an appropriately configured a syntax highlighting editor or IDE will generally show you which code sections are active and which are not.
#define DEBUG_ADCS_OBC
...
#if defined DEBUG_ADCS_OBC
printf("O2A ");
for (j = 0; j < 50; j++)
{
printf("%x ", buffer_obc[jj]);
}
#endif
I'll add one thing that didn't see being mentioned.
If optimizations are disabled on debug builds, and even if runtime performance impact is insignificant, code is still included. As a result debug builds are usually bigger than release builds.
If you have very limited memory, you can run into situation where release build fits in the device memory and debug build does not.
For this reason I prefer compile time #if over runtime if. I can keep the memory usage between debug and release builds closer to each other, and it's easier to keep using the debugger at the end of project.
The optimizer will solve the extra resources problem as mentioned in the other replies, but I want to add another point. From the code readability point of view this code will be repeated a lot of times, so you can consider creating your specific printing macros. Those macros is what should be enclosed by the debug enable or disable macros.
#ifdef DEBUG_DCS_OBS
myCustomPrint //your custom printing code
#else
myCustomPrint //No code here
#end
Also this will decrease the probability of the macro to be forgotten in any file which will cause a real optimization problem.

C++ variable address does not match

class CCtrl
{
...Other Members...
RankCache m_stRankCache;
uint32 m_uSyncListTime;
};
int CCtrl::UpdateList()
{
uint32 tNow = GetNowTime();
for (uint8 i = 0; i < uRankListNum; i++)
{
m_stRankCache.Append(i);
}
m_uSyncListTime = tNow;
return 0;
}
Here are two weired things:
When step into Append(), p this = 0x7f3f467edfdc, but in UpdateGuildList(), p &m_stRankCache = 0x7f3f067edfdc, these two pointers are different.
tNow = 1418916316, after executing m_uSyncListTime = tNow, m_uSyncListTime is still 0.
How could this happen? I've used a whole day for debugging. And I checked my code there is no pack(1) and pack() mismatch.
The issue is more than likely that you're using your debugger to debug code that has been optimized. As your comment suggested, you are debugging code that has been compiled with the -O3 flag, denoting optimization.
Even though you're using gdb, the Visual Studio and other debuggers also have the same issue, and that issue is debugging optimized code and having the debugger "work" in the sense that the debugger follows along with the lines in the source code, along with the variables that have been declared.
A debugger assumes that the lines of the source code match up with the generated assembly code. With optimizations turned on, this can no longer be the case. Code and variables are eliminated, moved, etc. Therefore the lines in the code (including variable declarations) you believe should be there at a certain location may not be there in the final optimized build.
The debugger cannot discern these changes, thus you get erroneous values used for variables, or in some cases, you get "variable doesn't exist" errors reported by your debugger.
Also, it may also serve as a good check to do a simple cout or log of the values in question if there is a problem with the debugging environment. There are situations where even debuggers may get things wrong, so a backup verification system (i.e. logging, printf() or cout statements, etc.) should be used.

Code littered with asserts

Hi I am programming on some device.
There is some sample with such code:
Verify(SomeFunc(argc, argv) == SDK_OK);
Verify(SomeOtherFunction(&st_initialData) == SDK_OK);
Verify(SomeOtherFunction2(x,y) == SDK_OK);
In doc, Verify is defined as 'similar' to assert.
My question is: if I build my project in Release mode, what will happen with above statements? Will they lose their power? Will the Verify have any effect still?
To avoid possible problem with above, will I have to replace above codes with checking return values like this?:
if(SomeFunc(argc, argv) == SDK_OK)
{
// we are fine
}
else
{
// handle error somehow, such that it is also available in Release mode
}
It is impossible to say, as it seems that it is your project which implements Verify, as a macro or as a function. Why don't you take a look at the implementation?
That being said, MFC framework has VERIFY macro which is similar to ASSERT, with the distinction that the expression is always evaluated, even in release build, but doesn't do anything if result of the expression is false. This might be the similar approach, as your examples seem to call some functions which can affect the system state.
I assume you mean the MFC VERIFY macro or something very similar.
Using this macro is safe for release builds. The argument is executed in any case, just the macro itself does nothing in release.
In contrast to this, the ASSERT macro is completely skipped in release builds, so the "side effects" of the argument do not happen. Therefore, VERIFY is used if the argument is required for the actual program flow, and ASSERT is used when the argument is for asserting only.
Almost certainly you will not have to replace them. If your project wanted just to remove the calls in production compilation, it would probably have just plain assert directly. Try to read the source of the project (always a good idea) and understand what he macro does.

How to properly rewrite ASSERT code to pass /analyze in msvc?

Visual Studio added code analysis (/analyze) for C/C++ in order to help identify bad code. This is quite a nice feature but when you deal with and old project you may be overwhelmed by the number of warnings.
Most of the problems are generating because the old code is doing some ASSERT at the beginning of the method or function.
I think this is the ASSERT definition used in the code (from afx.h)
#define ASSERT(f) DEBUG_ONLY((void) ((f) || !::AfxAssertFailedLine(THIS_FILE, __LINE__) || (AfxDebugBreak(), 0)))
Example code:
ASSERT(pBytes != NULL);
*pBytes = 0; // <- warning C6011: Dereferencing NULL pointer 'pBytes'
I'm looking for an easy, clean and safe solution to solve these warnings that does not imply disabling these warnings. Did I mention that there are lots of occurrences in current codebase?
/analyze is not guaranteed to yield relevant and correct warnings.
It can and will miss a lot of issues, and it also gives a number of false positives (things it identifies as warnings, but which are perfectly safe and will never actually occur)
It is unrealistic to expect to have zero warnings with /analyze.
It has pointed out a situation where you dereference a pointer which it can not verify is always valid. As far as PREfast can tell, there's no guarantee that it will never be NULL.
But that doesn't mean it can be NULL. Just that the analysis required to prove that it's safe is too complex for PREfast.
You may be able to use the Microsoft-specific extension __assume to tell the compiler that it shouldn't produce this warning, but a better solution is to leave the warning. Every time you compile with /analyze (which need not be every time you compile), you should verify that the warnings it does come up with are still false positives.
If you use your asserts correctly (to catch logic error during programming, guarding against situations that cannot happen, then I see no problem with your code, or with leaving the warning. Adding code to handle a problem that can never occur is just pointless. You're adding more code and more complexity for no reason (if it can never occur, then you have no way of recovering from it, because you have absolutely no clue what state the program will be in. All you know is that it has entered a code path you thought impossible.
However, if you use your assert as actual error handling (the value can be NULL in exceptional cases, you just expect that it won't happen), then it is a defect in your code. Then proper error handling (exceptions, typically) is needed.
Never ever use asserts for problems that are possible. Use them to verify that the impossible isn't happening. And when /analyze gives you warnings, look at them. If it is a false positive, ignore it (don't suppress it, because while it's a false positive today, the code you check in tomorrow may turn it into a real issue).
PREFast is telling you that you have a defect in your code; don't ignore it. You do in fact have one, but you have only skittered around acknowleging it. The problem is this: just because pBytes has never been NULL in development & testing doesn't mean it won't be in production. You don't handle that eventuality. PREfast knows this, and is trying to warn you that production environments are hostile, and will leave your code a smoking, mutilated mass of worthless bytes.
/rant
There are two ways to fix this: the Right Way, and a hack.
The right way is to handle NULL pointers at runtime:
void DoIt(char* pBytes)
{
assert(pBytes != NULL);
if( !pBytes )
return;
*pBytes = 0;
}
This will silence PREfast.
The hack is to use an annotation. For example:
void DoIt(char* pBytes)
{
assert(pBytes != NULL);
__analysis_assume( pBytes );
*pBytes = 0;
}
EDIT: Here's a link describing PREfast annotations. A starting point, anyway.
Firstly your assertion statement must guarantee to throw or terminate the application. After some experimentation I found in this case /analyse ignores all implementation in either template functions, inline functions or normal functions. You must instead use macros and the do{}while(0) trick, with inline suppression of
If you look at the definition of ATLENSURE() Microsoft use __analyse_assume() in their macro, they also have several paragraphs of very good documentation on why and how they are migrating ATL to use this macro.
As an example of this I have modified the CPPUNIT_ASSERT macros in the same way to clean up thousands of warnings in our unit tests.
#define CPPUNIT_ASSERT(condition) \
do { ( CPPUNIT_NS::Asserter::failIf( !(condition), \
CPPUNIT_NS::Message( "assertion failed" ), \
CPPUNIT_SOURCELINE() ) ); __analysis_assume(!!(condition)); \
__pragma( warning( push)) \
__pragma( warning( disable: 4127 )) \
} while(0) \
__pragma( warning( pop))
remember, ASSERT() goes away in a retail build so the C6011 warning is absolutely correct in your code above: you must check that pBytes is non-null as well as doing the ASSERT(). the ASSERT() simply throws your app into a debugger if that condition is met in a debug bug.
I work a great deal on /analyze and PREfast, so if you have other questions, please feel to let me know.
You seem to assume that ASSERT(ptr) somehow means that ptr is not NULL afterwards. That's not true, and the code analyzer doesn't make that assumption.
My co-author David LeBlanc would tell me this code is broken anyway, assuming you're using C++, you should use a reference rather than a pointer, and a ref can't be NULL :)