How to debug uninitialized variables in release mode in C++.
There's a warning for this. You should try to always compile cleanly at the highest warning level. For VC++ this is level 4. Turn off specific annoying warnings selectively only.
Also, unless you deliberately uncheck the option, VC++ will compile with /RTCu (or even /RTCsu) which puts in checks to catch uninitialized variables at run-time.
Of course, proper programming style (introduce variables as late as possible) will prevent such errors from happening in the first place.
Generally, rather than debugging uninitialized variables, you want to prevent the very possibility, such as using classes/objects with ctors, so creating one automatically and unavoidably initializes it.
When you do use something like an int, it should generally be initialized as it's created anyway, so uninitialized variables will be pretty obvious from simple inspection (and you generally want to keep your functions small enough that such inspection is easy).
Finally, most decent compilers can warn you about at least quite a few attempts at using variables without initialization. Clearly such warnings should always be enabled. One important point: these often depend on data-flow analysis that's intended primarily for optimization, so many compilers can/will only issue such warnings when you enable at least some degree of optimization.
I don't know about VC++, but for gcc, there is a warning option -Wuninitialized that can be used while compiling. Details: http://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
Append: -Wuninitialized is included in -Wall, i.e warn all, one of the recommended and most used warning flag. In addition, having -Werror would fail the compilation whenever any such warning arises.
Uninitialized variables are a nasty bug to find. Some static checkers would probably be able to find your uninitialized variable. There are open source ones. You might be able to get a trial version of commercial version as well.
If you do not have debugger, you need to add logging statements in your code wherever you want to see the values of variables which you suspect uninitialized.
Sometimes, logging statement may lead to crash if passed an uninitialized pointer. So you can catch the bug there itself in this case.
You need to build release binaries with debug symbols. Here is a reference that may be helpful if you are on Visual Studio.
There must be something analogous for other implementations as well.
Use something like CPPcheck (open-source) or PC-Lint (commercial) to check for them. They will help find a lot of other errors.
Related
To find out all the (possible) problems that existed in the program, we had better turn on all the debug tools of the compiler. The tool will always tell us something like "remark #7712: This variable has not been used.".
In many cases, in order to keep some rules, I have to keep some input and output without using them. At the same time, I want to keep the debug tool turned on.
Can we do something by standard grammar to tell the compiler we really mean to do it and do not report any warning about it?
The Fortran standard sets out the rules for correct programs and requires that compilers identify any breach of those rules. Such breaches, which cause compilation to fail, are generally known as errors.
However, programmers make many mistakes which are not errors and which a (Fortran) compiler is not required to spot. Some compilers provide additional diagnostic capabilities, such as identifying unused variables, which go beyond what the standard requires. The compilers raise what are generally known as warnings in these cases. This type of mistake does not cause compilation to fail. Compilers also generally provide some means to determine which warnings are raised during compilation, so that you can switch off and on this diagnostic capability. For details of these capabilities refer to your compiler's documentation.
The standard is entirely silent on this type of mistake so, if I understand the question correctly, there is nothing
by standard grammar to tell the compiler we really mean to do it and
do not report any warning about it
The simplest thing (besides of course not declaring things you don't use)
may be to simply use the variables.
real x
x=huge(x) !reminder x is declared but not used.
at least makes gfortran happy that you have "used" the variable.
I would like advice how to proceed in such situation.
Imagine I have large C++ project which works well.
I have suspicion there might be some UB in this code (because in different project written by same author I found UB).
Now, say I need to add new features to this project.
I am afraid because:
if I recompile with new compiler this can increase risk of UB happening if in the code is UB already. (e.g. new compiler might not be OK with UB which the old compiler was fine with).
Is it realistic to eliminate all UB in this large project by eye inspection (before I move to adding new feature)??
If not, then I should at least compile with same version of compiler right? (to decrease chance of problems if there is UB).
Project is done in Visual Studio so I don't know if there are object files, in which case, I could leave object files same and only modify parts in files where I need to add something - thus again minimizing risk of UB.
What is the course of action in such situation? I think this could be pretty common scenario.
I like suggestion that I test the project using new compiler before adding new code, but even then - we know testing might not reveal UB, isn't it?
In order, I would:
Compile with -Wall (/W4 for you Windows folk) and fix errors.
Write tests if there aren't any already.
Use tools like valgrind to detect issues and fix them.
Study synchronization primitives if in use, and use modern paradigms where possible.
Document the code and adhere to a style guide.
I would not attempt to avoid problems by keeping object files around. That's a nightmarish maintenance problem.
Undefined Behavior = Bugs
It's impossible to prove that a project is bug-free. Even the best programmers do create bugs. Even the best code-review cannot eliminate all bugs in a project. No, it's not realistic to eliminate all UB in a project of some size by code inspection or by any other means. Your best option is to review the code and eliminate as many as possible.
Change your perception of UB (bugs): If you encounter a bug during your re-engineering efforts, it's a good thing! You are in the best position to remove one UB.
Don't keep the old compiler just because you are afraid of UB. Recompile the project with the latest and best compiler available. Compilers can also have bugs. Newer compilers will produce better, more robust code. Newer compilers will produce better warnings. Use all warnings possible -Wall.
Eliminate all the warnings that the compiler produces. Every single warning is there for a reason, it highlights a problem. The likelihood of a "false positive" is quite dim nowadays. This is even true for MSVC (I'm not talking about real old compilers like before VC 2005)
Use a static code checker (Cppcheck). It can point you to common problems with the code.
Use a custom rule set for your code checker. It will help you to get the code up to some standard.
If possible, compile the project with another compiler (GCC, Clang) just for the sake of getting the warnings of these compilers.
Don't link against old object files. This will create more problems than what you think it avoids
As others said: First and foremost, try to find the errors, not hide them.
The first and simplest measure is to set the warning level to /W4 (you can try Wall, but due to the large amount of noise this will produce (e.g. from standard headerfiles), it is usually only of help if you know you have an error in a certain part of your code)
Use static analyzers - you can start with the builtin Code Analysis tool and then go for external tools (which are usually much more difficult to set up correctly for a non-trivial project).
Write lots of tests and make sure, you are exercising edge cases - thats where UB usually lurks.
If possible, try to compile the project (or parts of it) under clang and activate the different sanitizers (in particular there is UndefinedBehaviorSanitizer) which will further instrument your code to check for UB (only helpfull if you have tests to exercise that UB though)
Test your code at different optimization levels and combination of flags (in VS, especially _ITERATOR_DEBUG_LEVEL can be helpfull to find out-of-bounds errors)
I'd say any non-trivial code base potentially contains undefined behavior. What is special about that particular Programmer? If he/she is prone to a special kind of UB, then you can focus your efforts on this.
I'm writing a program in C++/Qt which contains a graph file parser. I use g++ to compile the project.
While developing, I am constantly comparing the performance of my low level parser layer between different compiler flags regarding optimization and debug information, plus Qt's debug flag (turning on/off qDebug() and Q_ASSERT()).
Now I'm facing a problem where the only correctly functioning build is the one without any optimization. All other versions, even with -O1, seem to work in another way. They crash due to unsatisfied assertions, which are satisfied when compiled without a -O... flag. The code doesn't produce any compiler warning, even with -Wall.
I am very sure that there is a bug in my program, which seems to be only harmful with optimization being enabled. The problem is: I can't find it even when debugging the program. The parser seems to read wrong data from the file. When I run some simple test cases, they run perfectly. When I run a bigger test case (a route calculation on a graph read directly from a file), there is an incorrect read in the file which I can't explain.
Where should I start tracking down the problem of this undefined behavior? Which optimization methods are possibly involved within this different behavior? (I could enable all flags one after the other, but I don't know that much compiler flags but -O... and I know that there are a lot of them, so this would need a very long time.) As soon as I know which type the bug is of, I am sure I find it sooner or later.
You can help me a lot if you can tell me which compiler optimization methods are possible candidates for such problems.
There are a few classes of bugs that commonly arise in optimized builds, that often don't arise in debug builds.
Un-initialized variables. The compiler can catch some but not all. Look at all your constructors, look at global variables. etc. Particularly look for uninitialized pointers. In a debug build memory is reset to zero, but in a release build it isn't.
Use of temporaries that have gone out of scope. For example when you return a reference to a local temporary in a function. These often work in debug builds because the stack is padded out more. The temporaries tend to survive on the stack a little longer.
array overruns writing of temporaries. For example if you create an array as a temporary in a function and then write one element beyond the end. Again, the stack will have extra space in debug ( for debugging information ) and your overrun won't hit program data.
There are optimizations you can disable from the optimized build to help make debugging the optimized version easier.
-g -O1 -fno-inline -fno-loop-optimize -fno-if-conversion -fno-if-conversion2 \
-fno-delayed-branch
This should make stepping through your code in the debugger a little easier to follow.
Another suggestion is that if the assertions you have do not give you enough information about what is causing the problem, you should consider adding more assertions. If you are afraid of performance issues, or assertion clutter, you can wrap them in a macro. This allows you to distinguish the debugging assertions from the ones you originally added, so they can be disabled or removed from your code later.
1) Use valgrind on the broken version. (For that matter, try valgrind on the working version, maybe you'll get lucky.)
2) Build the system with "-O1 -g" and step through your program with gdb. At the crash, what variable has an incorrect value? Re-run your program and note when that variable is written to (or when it isn't and should have been.)
When compiler optimizes code, what is the scope of the optimizations? Can an optimization
a) span more than a set of nested braces?
b) span more than a function?
c) span more than a file?
We are chasing an obscure bug that seems to derive from optimization. Code crashes in release mode but not debug mode. And of course we are aware this could be heap corruption or other memory problem bur have been chasing this for a while now. One avenue we are considering is to selectively compile our files in debug mode until the problem goes away. In other words:
a) Start with all file compiled in release mode
b) Compile 1/2 the files in debug mode
if crash still seen, take half the release compiled files and compile in debug mode
if crash not seen, talk half the files compiled in debug mode and compile in release mode
repeat until we narrow in on suspect files
this is a binary search to narrow in on problem files
We are aware that if this is a memory issue, simple doing this mixed compilation may make the bug go away, but we are curous if we can narrow in on the problem files.
The outstanding question though is what is the scope of optimizations - can they span more than one file?
An optimization can do literally anything as long as it doesn't change the semantics of the behaviour defined by the language. That means the answers to your first questions (a), (b), and (c) are all yes. In practice, most compilers aren't that ambitious, but there are certainly some examples. Clang and LLVM have flags for link time optimization that allow optimizations to span pretty much the entire program. MSVC has a similar /GL flag that allows whole-program optimization.
Often the causes of these sorts of failures are uninitialized variables. Static analysis tools can be very helpful in finding problems like the one you're describing. I'm not sure your binary searching via optimizations is going to help you track down too much, though it is possible. Your bug could be the result of pretty complicated interactions between modules.
Good luck!
You can approximately identify problem files from call traces of crash in release mode. Then try to rebuild them without optimizations - this is much simpler than binary search.
To know about compiler optimization, one easy resourse could be wikipedia. It nicely explains in brief some important and wide-implemented optimizations by almost all modern compilers.
Maybe, you would like to read the wiki entry first, especially these sections:
Types of optimizations
Specific techniques
I think you are asking the wrong question.
It is unlikely (unless you are doing something special) that the optimizer is your problem.
The problem is most likely an uninitialized variable in your code.
When you compile in debug mode most compilers will initialize all memory to a specific pattern, so that during debugging you can see what has happened to the memory (was it allocated/ was it de-allcoated/ was it from the stack/ Was the stack popped back from etc).
Thus in debug mode any uninitialized memory has the a specific pattern that can be recognized by the debugger. The exact pattern will depend on the compiler. But this can make sloppy code work as the uninitialized pointers are NULL, integer counters start at 0 etc.
When you compile for release mode all the extra initialization is turned off. If you did not explicitly initialize the memory it is in a random state. This is what debug/release versions of an application behave differently.
The easy way to check for this is to check your compiler warnings.
Make sure that all variables are initialized before use (it will be in the warnings).
Many times I work with optimized code (sometimes even involving vectorized loops), which contain bugs and such. How would one debug such code? I'm looking for any kind of tools or techniques. I use the following (possibly outdated) tools, so I'm looking to upgrade.
I use the following:
Since with ddd, you cannot see the code, I use gdb+ dissambler command and see the produced code; I can't really step through the program using this.
ndisasm
Thanks
It is always harder to debug optimised programs, but there are always ways. Some additional tips:
Make a debug build, and see if you get the same bug in a debug build. No point debugging an optimised version if you don't have to.
Use valgrind if on a platform that supports it. The errors you see may be harder to understand, but catching the problem early often simplifies debugging.
printf debugging is primitive, but sometimes it is the simplest way if you have a complex issue that only shows up in optimised builds.
If you suspect a timing issue (especially in a multithreaded program), roll your own version of assert which aborts or prints if the condition is violated, and use it in a few select places, to rule out possible problems.
See if you can reproduce the problem without using -fomit-frame-pointers, since that makes code very hard to debug, and with -O2 or -O3 enabled. That might give you enough information to find the cause of your problem.
Isolate parts of your code, build a test-suite, and see if you can identify any testcases which fail. It is much easier to debug one function than the whole program.
Try turning off optimisations one by one with the -fno-X options. This might help you find common problems like strict aliasing problems.
Turn on more compiler warnings. Some things, like strict aliasing problems, can generate compiler warnings if they create a difference in behaviour between different optimisation levels.
When debugging release builds you can put in __asm nops; as a placeholder for breakpoints (int 3). This is nice as you can guarantee breakpoint locations without messing up compiler optimizations or writing printf/cout statements.
It's always easier to debug a non-optimized version, of course. Failing that, disassembly of the code can be helpful. Other techinques I've used include partially de-optimizing the code by forcing intermediate results to be printed or logged, or changing a critical variable to "volatile" so I can at least look at that value in the debugger.
Chances are what you call optimized code is scrambled to shave cycles (which makes debugging hard) but is not really very optimized. Here is an example of what I mean.
I would turn off the compiler optimization, debug and tune it yourself, and then turn compiler optimization back on if the code has hotspots that are actually in code the compiler sees (not in outside libraries). (I define a hotspot as a part of code where the PC is often found. That automatically exempts loops containing function calls because they steal away the PC.)