Debug session freezes when trying to watch an array - c++

I am using ubuntu 12.04. I have used so far anjuta and codelite as IDE's for C++ school projects.
However, with both of them I have encountered one problem:
After starting the debugger, everything works fine till I try to add an array at watches' section. It does not display anything and when I try to continue debugging it freezes and I have to stop the debug session. I have to mention that watching variables works well.
Thank you,
LE: Actually, the debug function freezes only in case of large arrays...it may be a bug of codelite then. Any opinion?

I have to mention that watching variables works well.
When you set the watchpoint on a variable, GDB probably says Hardware watchpoint N (but your IDE may be hiding that message).
When you set a watchpoint on anything larger than 8 bytes on x86 processor, GDB can not set a hardware watchpoint (because x86 hardware doesn't support such watchpoints). GDB sets a software watchpoint instead. Software watchpoints are implemented as follows:
single-step the program
did values change? No -> go to step 1. Yes: stop.
Software watchpoints are really slow. If you watch your system with top, you'll likely discover that GDB is consuming 100% CPU.
If you really need to watch an entire array, this answer shows how that can be done with valgrind.

Related

gdb freezing when print variables with tab completion

I compile the c++ project, which is not too large, about 6M binary. When I debug it and want to print some variable, I type the first two characters and press the Tab to complete. Then the gdb read symbols forever freezing. How can I solve this problem. thank you!
I type the first two characters and press the Tab to complete. Then the gdb read symbols forever freezing. How can I solve this problem
Doctor, it hurts when I do that.
Well, don't do that.
Seriously, if you have a very large binary (it's unclear whether your 6MB is the size with debug info or without), and lots of variables, then GDB will necessarily have to spend some time searching for variables matching your two initial characters.
That said,
we routinely debug binaries that are 2GB in size or larger, and
have spent quite a lot of effort improving GDB experience with such binaries
So perhaps your first step should be to take the latest release of GDB, and see if the problem has already been solved for you.
Update:
My binary is 6MB with debug info
That's not large at all. Certainly it should not cause more than a few seconds delay to list all variables in such a binary.
My GDB version is "GNU gdb (GDB) 7.6.2"
That's the latest release.
It's probably safe to conclude that there is a bug in GDB.
If you can construct a minimal test case that shows the problem, then your best bet is to report it as a bug in http://sourceware.org/bugzilla.
If you can't, you'll have to debug GDB yourself. A reasonable place to start is running strace -p <pid-of-hung-gdb> and gdb -p <pid-of-hung-gdb>; (gdb) where to find out exactly where GDB is getting stuck.
If you can update to GDB 7.10, your tab-completion freeze-ups should disappear.
GDB 7.10 (as of August 2015) contains a feature to address this problem.
set max-completions
Set the maximum number of candidates to be considered during
completion. The default value is 200. This limit allows GDB to avoid
generating large completion lists, the computation of which can cause
the debugger to become temporarily unresponsive.
[The above quote is taken from the patch shown on the gitweb site for gdb]
The GDB news release lists the feature as: "The number of candidates to be considered during completion can now be limited."
Updating to GDB 7.10 solved the problem for me. The default value of 200 for max-completions was sufficient. I did not customize it.

how does gdb work?

I want to know how does gdb work internally.
e.g. I know a brief idea that it makes use of ptrace() system call to monitor traced program.
But I want to know how it handles signals, how it inserts new code, and other such fabulous things it does.
Check out the GDB Internals Manual, which covers some of the important aspects. There's also an older PDF version of this document.
From the manual:
This document documents the internals of the GNU debugger, gdb. It includes description of gdb's key algorithms and operations, as well as the mechanisms that adapt gdb to specific hosts and targets.
Taken from gdbint.pdf:
It can be done either as hardware breakpoints or as software
breakpoints:
Hardware breakpoints are sometimes available as a builtin debugging features with some chips. Typically these work by having dedicated
register into which the breakpoint address may be stored. If the PC
(shorthand for program counter) ever matches a value in a breakpoint
registers, the CPU raises an exception and reports it to GDB.
Another possibility is when an emulator is in use; many emulators include circuitry that watches the address lines coming out from the
processor, and force it to stop if the address matches a breakpoint's
address.
A third possibility is that the target already has the ability to do breakpoints somehow; for instance, a ROM monitor may do its own
software breakpoints. So although these are not literally hardware
breakpoints, from GDB's point of view they work the same;
Software breakpoints require GDB to do somewhat more work. The basic theory is that GDB will replace a program instruction with a trap,
illegal divide, or some other instruction that will cause an
exception, and then when it's encountered, GDB will take the exception
and stop the program. When the user says to continue, GDB will restore
the original instruction, single-step, re-insert the trap, and
continue on.
The only way you'll find out is by studying the source.
You can also build it and debug it with itself. Step through the code, and you'll know exactly how it does what it does.
Reading GDB source is not for the faint of heart though -- it is chock-full of macros, and heavily uses libbfd, which itself is hard to understand.
It has to, because it is portable (and in particular, builds and works on platforms which do not have ptrace() at all).

What causes a Sigtrap in a Debug Session

In my c++ program I'm using a library which will "send?" a Sigtrap on a certain operations when
I'm debugging it (using gdb as a debugger). I can then choose whether I wish to Continue or Stop the program. If I choose to continue the program works as expected, but setting custom breakpoints after a Sigtrap has been caught causes the debugger/program to crash.
So here are my questions:
What causes such a Sigtrap? Is it a leftover line of code that can be removed, or is it caused by the debugger when he "finds something he doesn't like" ?
Is a sigtrap, generally speaking, a bad thing, and if so, why does the program run flawlessly when I compile a Release and not a Debug Version?
What does a Sigtrap indicate?
This is a more general approach to a question I posted yesterday Boost Filesystem: recursive_directory_iterator constructor causes SIGTRAPS and debug problems.
I think my question was far to specific, and I don't want you to solve my problem but help me (and hopefully others) to understand the background.
Thanks a lot.
With processors that support instruction breakpoints or data watchpoints, the debugger will ask the CPU to watch for instruction accesses to a specific address, or data reads/writes to a specific address, and then run full-speed.
When the processor detects the event, it will trap into the kernel, and the kernel will send SIGTRAP to the process being debugged. Normally, SIGTRAP would kill the process, but because it is being debugged, the debugger will be notified of the signal and handle it, mostly by letting you inspect the state of the process before continuing execution.
With processors that don't support breakpoints or watchpoints, the entire debugging environment is probably done through code interpretation and memory emulation, which is immensely slower. (I imagine clever tricks could be done by setting pagetable flags to forbid reading or writing, whichever needs to be trapped, and letting the kernel fix up the pagetables, signaling the debugger, and then restricting the page flags again. This could probably support near-arbitrary number of watchpoints and breakpoints, and run only marginally slower for cases when the watchpoint or breakpoint aren't frequently accessed.)
The question I placed into the comment field looks apropos here, only because Windows isn't actually sending a SIGTRAP, but rather signaling a breakpoint in its own native way. I assume when you're debugging programs, that debug versions of system libraries are used, and ensure that memory accesses appear to make sense. You might have a bug in your program that is papered-over at runtime, but may in fact be causing further problems elsewhere.
I haven't done development on Windows, but perhaps you could get further details by looking through your Windows Event Log?
While working in Eclipse with minGW/gcc compiler, I realized it's reacting very bad with vectors in my code, resulting to an unclear SIGTRAP signal and sometimes even showing abnormal debugger behavior (i.e. jumping somewhere up in the code and continuing execution of the code in reverse order!).
I have copied the files from my project into the VisualStudio and resolved the issues, then copied the changes back to eclipse and voila, worked like a charm. The reasons were like vector initialization differences with reserve() and resize() functions, or trying to access elements out of the bounds of the vector array.
Hope this will help someone else.
I received a SIGTRAP from my debugger and found out that the cause was due to a missing return value.
string getName() { printf("Name!");};

GDB hardware watchpoint very slow - why?

On a large C application, I have set a hardware watchpoint on a memory address as follows:
(gdb) watch *0x12F5D58
Hardware watchpoint 3: *0x12F5D58
As you can see, it's a hardware watchpoint, not software, which would explain the slowness.
Now the application running time under debugger has changed from less than ten seconds to one hour and counting. The watchpoint has triggered three times so far, the first time after 15 minutes when the memory page containing the address was made readable by sbrk. Surely during those 15 minutes the watchpoint should have been efficient since the memory page was inaccessible? And that still does not explain, why it's so slow afterwards.
The platform is x86_64 and the GDB versions are Ubuntu 9.10 package:
$ gdb --version
GNU gdb (GDB) 7.0-ubuntu
[...]
and stock GDB 7.1 built from sources:
$ gdb-7.1 --version
GNU gdb (GDB) 7.1
Thanks in advance for any ideas as what might be the cause or how to fix/work around it.
EDIT: removed cast
EDIT: gdb 7.1
I discovered that watching a large character buffer was very slow, whereas watching a character in that buffer was very fast.
e.g.
static char buf[1024];
static char* buf_address = &buf;
watch buf_address - excruciatingly slow.
watch *buf_address - very fast.
I've actually had trouble with hardware watchpoints in GDB 7.x.x., which is not acceptable since watchpoints are a necessity in my job.
On advice from a co-worker, I downloaded the source for 6.7.1 and built it locally. Watchpoints work much better now.
Might be worth a try.
It's most likely because you're casting it each time. Try this:
(gdb) watch *0x12F5D58
Another option is that you have too many hardware watchpoints set, so gdb is forced to use software watchpoints. Try checking how many watchpoints you have using:
(gdb) info break
and see if you can disable some watchpoints.
On x86 you have the following limitation: all your watchpoints can cover no more than four memory addresses, each address of memory can watch for one memory word - this is because hardware watchpoints (the fast ones) use the processors debug registers, an you have four of them, therefore four locations to watch for.

What exactly does a debugger do?

I've stumbled onto a very interesting issue where a function (has to deal with the Windows clipboard) in my app only works properly when a breakpoint is hit inside the function. This got me wondering, what exactly does the debugger do (VS2008, C++) when it hits a breakpoint?
Without directly answering your question (since I suspect the debugger's internal workings may not really be the problem), I'll offer two possible reasons this might occur that I've seen before:
First, your program does pause when it hits a breakpoint, and often that delay is enough time for something to happen (perhaps in another thread or another process) that has to happen before your function will work. One easy way to verify this is to add a pause for a few seconds beforehand and run the program normally. If that works, you'll have to look for a more reliable way of finding the problem.
Second, Visual Studio has historically (I'm not certain about 2008) over-allocated memory when running in debug mode. So, for example, if you have an array of int[10] allocated, it should, by rights, get 40 bytes of memory, but Visual Studio might give it 44 or more, presumably in case you have an out-of-bounds error. Of course, if you DO have an out-of-bounds error, this over-allocation might make it appear to be working anyway.
Typically, for software breakpoints, the debugger places an interrupt instruction at the location you set the breakpoint at. This transfers control of the program to the debugger's interrupt handler, and from there you're in a world where the debugger can decide what to do (present you with a command prompt, print the stack and continue, what have you.)
On a related note, "This works in the debugger but not when I run without a breakpoint" suggests to me that you have a race condition. So if your app is multithreaded, consider examining your locking discipline.
It might be a timing / thread synchronization issue. Do you do any multimedia or multithreading stuff in your program?
The reason your app only works properly when a breakpoint is hit might be that you have some watches with side effects still in your watch list from previous debugging sessions. When you hit the break point, the watch is executed and your program behaves differently.
http://en.wikipedia.org/wiki/Debugger
A debugger essentially allows you to step through your source code and examine how the code is working. If you set a breakpoint, and run in debug mode, your code will pause at that break point and allow you to step into the code. This has some distinct advantages. First, you can see what the status of your variables are in memory. Second, it allows you to make sure your code is doing what you expect it to do without having to do a whole ton of print statements. And, third, it let's you make sure the logic is working the way you expect it to work.
Edit: A debugger is one of the more valuable tools in my development toolbox, and I'd recommend that you learn and understand how to use the tool to improve your development process.
I'd recommend reading the Wikipedia article for more information.
The debugger just halts execution of your program when it hits a breakpoint. If your program is working okay when it hits the breakpoint, but doesn't work without the breakpoint, that would indicate to me that you have a race condition or another threading issue in your code. The breakpoint is stopping the execution of your code, perhaps allowing another process to complete normally?
It stops the program counter for your process (the one you are debugging), and shows the current value of your variables, and uses the value of your variables at the moment to calculate expressions.
You must take into account, that if you edit some variable value when you hit a breakpoint, you are altering your process state, so it may behave differently.
Debugging is possible because the compiler inserts debugging information (such as function names, variable names, etc) into your executable. Its possible not to include this information.
Debuggers sometimes change the way the program behaves in order to work properly.
I'm not sure about Visual Studio but in Eclipse for example. Java classes are not loaded the same when ran inside the IDE and when ran outside of it.
You may also be having a race condition and the debugger stops one of the threads so when you continue the program flow it's at the right conditions.
More info on the program might help.
On Windows there is another difference caused by the debugger. When your program is launched by the debugger, Windows will use a different memory manager (heap manager to be exact) for your program. Instead of the default heap manager your program will now get the debug heap manager, which differs in the following points:
it initializes allocated memory to a pattern (0xCDCDCDCD comes to mind but I could be wrong)
it fills freed memory with another pattern
it overallocates heap allocations (like a previous answer mentioned)
All in all it changes the memory use patterns of your program so if you have a memory thrashing bug somewhere its behavior might change.
Two useful tricks:
Use PageHeap to catch memory accesses beyond the end of allocated blocks
Build using the /RTCsu (older Visual C++ compilers: /GX) switch. This will initialize the memory for all your local variables to a nonzero bit pattern and will also throw a runtime error when an unitialized local variable is accessed.