I have the following situation: My program runs fine, does all I want it to do, tests pass and Valgrind says it's all right. The only issue is the fact that if I pause execution at some point and try to inspect the state of some objects in the debug view of QtCreator (using GDB) some variables become <not-accessible> and, on resuming execution it reaches a segmentation fault.
To be a little more specific, the program is single threaded and this happens while following the pointers in a tree structure. The structure seems to be fine by the output of the tests.
Does anyone know about a possible cause? Maybe I have messed up with the stack in a way that luckily doesn't affect the tests or it may be just an IDE or debugger issue that should not care about? Thanks in advance for any answers.
Does anyone know about a possible cause?
Do you have multiple threads in your program?
When a program behaves differently depending on whether some GDB breakpoints are present or not, in 99.99% of the cases the program has a data race, and mere fact of stopping it at "inopportune" time exposes that fact.
On Linux, you could use Thread Sanitizer to check for data races.
Related
I'm struggling to debug a multithreaded c++ program that I am working on. It seems to be crashing at the same spot in the code, but only some of the time due to a variable being deallocated by a different thread, causing the program to crash when the variable is accessed by a pointer after that.
Because of this, there is no way for me to step through the program for debugging. Is there any way to track all instances of a specific variable in gdb, and track what specific actions are performed to that memory location (so I can see who deallocated it, and where)? Or are there any other good ways to debug this?
I'm struggling to debug a multithreaded c++ program that I am working on
Debugging multithreaded programs with GDB (or any other debugger) is hard.
For the specific example you gave: one thread free()ing memory that another thread is still accessing, Valgind and AddressSanitizer will likely yield much better result than GDB.
I'm writing a C++ Software Image processing tool.The tool works fine, but suddenly it stops and it never sends any exception or any crash or nothing that can let me which line or which area that does that crash.
How can I determine that faulty code in that situation?
There's a few things you can do:
First of all though, it sounds more like an infinite loop, deadlock, or like you're using all of your system resources and it's just slowing down and taking a very (possibly infinite) long time. If that's the case, you'll have to find it with debugging.
Things that you can try - not necessarily in this order:
Look for shared variables you're using. Is there a chance that you
have a deadlock with threads and mutexes? Think about it and try to
fix it.
Check for use of uninitialized variables/pointers. Sometimes
(rarely) you can get very strange behavior when you invoke undefined
behavior - I'm not a windows C++ dev (I work on Linux), but it
wouldn't be the first time I saw a lockup from a segmentation fault.
Add error output (std::cerr/stderror) to your processing logic so
you can see how far in it crashes. After that, set a condition to
catch it around that point so you can watch it happen in the
debugger and see the state of your variables and what might be
wrong.
Do a stack trace so you can see what calls have been hit most
recently. This will at least let you know the last function chain
that executed.
Have you used stack tracing before?
Look up MSDN documentation on how to use them. They have different type of stack trace depending on your application.
You could
Use a debugger
Add logging code to your program
Cut sections of code from your program until it starts working
Start with the first.
I'm currently working on a program (in C++, using Code::Blocks) that uses a lot of random numbers and takes a while to get going; most of the time, it works fine, but every now and then it performs an illegal operation and must shut down. Given the random numbers all over the place, and the fact that it currently takes ~3-5 minutes for the program to reach the stage at which the errors occur (this timeframe is normal/acceptable), reproducing the problems reliably and convenient is extremely difficult, and reporting on every other line of code to cout to manually track things is time-consuming, visually clutters reporting on things not related to bugs, and is not always helpful, since even if I know when the program stops, I sometimes don't know why.
Is there some way for me to see what the last operation in the program was before it crashed, and for me to see why this operation lead to a crash? Something within CodeBlocks would be best, but something third-party works too. It also needs to be something I can use every time I test the program, because I never know when a crash is going to occur.
That is what debuggers are for. Build the system with full debugging symbols, configure the system so that you get a full crash report (in linux a core file), and then launch the debugger with the core file (alternatively run the whole program inside the debugger, but that might take a while, running inside a debugger is usually much slower than running outside of it.
The debugger should be able to give you the state of the program when the illegal instruction happened and you will get some insight as of the state that the program was on. From there either you figure what is wrong, or maybe you can make a couple of smaller testcases that might trigger the error.
Debugging issues that cannot be reproduced systematically is a pain, good luck there!
Sounds like you want a debugger. Debugging C and C++ programs using GDB
In my c++ program I'm using a library which will "send?" a Sigtrap on a certain operations when
I'm debugging it (using gdb as a debugger). I can then choose whether I wish to Continue or Stop the program. If I choose to continue the program works as expected, but setting custom breakpoints after a Sigtrap has been caught causes the debugger/program to crash.
So here are my questions:
What causes such a Sigtrap? Is it a leftover line of code that can be removed, or is it caused by the debugger when he "finds something he doesn't like" ?
Is a sigtrap, generally speaking, a bad thing, and if so, why does the program run flawlessly when I compile a Release and not a Debug Version?
What does a Sigtrap indicate?
This is a more general approach to a question I posted yesterday Boost Filesystem: recursive_directory_iterator constructor causes SIGTRAPS and debug problems.
I think my question was far to specific, and I don't want you to solve my problem but help me (and hopefully others) to understand the background.
Thanks a lot.
With processors that support instruction breakpoints or data watchpoints, the debugger will ask the CPU to watch for instruction accesses to a specific address, or data reads/writes to a specific address, and then run full-speed.
When the processor detects the event, it will trap into the kernel, and the kernel will send SIGTRAP to the process being debugged. Normally, SIGTRAP would kill the process, but because it is being debugged, the debugger will be notified of the signal and handle it, mostly by letting you inspect the state of the process before continuing execution.
With processors that don't support breakpoints or watchpoints, the entire debugging environment is probably done through code interpretation and memory emulation, which is immensely slower. (I imagine clever tricks could be done by setting pagetable flags to forbid reading or writing, whichever needs to be trapped, and letting the kernel fix up the pagetables, signaling the debugger, and then restricting the page flags again. This could probably support near-arbitrary number of watchpoints and breakpoints, and run only marginally slower for cases when the watchpoint or breakpoint aren't frequently accessed.)
The question I placed into the comment field looks apropos here, only because Windows isn't actually sending a SIGTRAP, but rather signaling a breakpoint in its own native way. I assume when you're debugging programs, that debug versions of system libraries are used, and ensure that memory accesses appear to make sense. You might have a bug in your program that is papered-over at runtime, but may in fact be causing further problems elsewhere.
I haven't done development on Windows, but perhaps you could get further details by looking through your Windows Event Log?
While working in Eclipse with minGW/gcc compiler, I realized it's reacting very bad with vectors in my code, resulting to an unclear SIGTRAP signal and sometimes even showing abnormal debugger behavior (i.e. jumping somewhere up in the code and continuing execution of the code in reverse order!).
I have copied the files from my project into the VisualStudio and resolved the issues, then copied the changes back to eclipse and voila, worked like a charm. The reasons were like vector initialization differences with reserve() and resize() functions, or trying to access elements out of the bounds of the vector array.
Hope this will help someone else.
I received a SIGTRAP from my debugger and found out that the cause was due to a missing return value.
string getName() { printf("Name!");};
I've stumbled onto a very interesting issue where a function (has to deal with the Windows clipboard) in my app only works properly when a breakpoint is hit inside the function. This got me wondering, what exactly does the debugger do (VS2008, C++) when it hits a breakpoint?
Without directly answering your question (since I suspect the debugger's internal workings may not really be the problem), I'll offer two possible reasons this might occur that I've seen before:
First, your program does pause when it hits a breakpoint, and often that delay is enough time for something to happen (perhaps in another thread or another process) that has to happen before your function will work. One easy way to verify this is to add a pause for a few seconds beforehand and run the program normally. If that works, you'll have to look for a more reliable way of finding the problem.
Second, Visual Studio has historically (I'm not certain about 2008) over-allocated memory when running in debug mode. So, for example, if you have an array of int[10] allocated, it should, by rights, get 40 bytes of memory, but Visual Studio might give it 44 or more, presumably in case you have an out-of-bounds error. Of course, if you DO have an out-of-bounds error, this over-allocation might make it appear to be working anyway.
Typically, for software breakpoints, the debugger places an interrupt instruction at the location you set the breakpoint at. This transfers control of the program to the debugger's interrupt handler, and from there you're in a world where the debugger can decide what to do (present you with a command prompt, print the stack and continue, what have you.)
On a related note, "This works in the debugger but not when I run without a breakpoint" suggests to me that you have a race condition. So if your app is multithreaded, consider examining your locking discipline.
It might be a timing / thread synchronization issue. Do you do any multimedia or multithreading stuff in your program?
The reason your app only works properly when a breakpoint is hit might be that you have some watches with side effects still in your watch list from previous debugging sessions. When you hit the break point, the watch is executed and your program behaves differently.
http://en.wikipedia.org/wiki/Debugger
A debugger essentially allows you to step through your source code and examine how the code is working. If you set a breakpoint, and run in debug mode, your code will pause at that break point and allow you to step into the code. This has some distinct advantages. First, you can see what the status of your variables are in memory. Second, it allows you to make sure your code is doing what you expect it to do without having to do a whole ton of print statements. And, third, it let's you make sure the logic is working the way you expect it to work.
Edit: A debugger is one of the more valuable tools in my development toolbox, and I'd recommend that you learn and understand how to use the tool to improve your development process.
I'd recommend reading the Wikipedia article for more information.
The debugger just halts execution of your program when it hits a breakpoint. If your program is working okay when it hits the breakpoint, but doesn't work without the breakpoint, that would indicate to me that you have a race condition or another threading issue in your code. The breakpoint is stopping the execution of your code, perhaps allowing another process to complete normally?
It stops the program counter for your process (the one you are debugging), and shows the current value of your variables, and uses the value of your variables at the moment to calculate expressions.
You must take into account, that if you edit some variable value when you hit a breakpoint, you are altering your process state, so it may behave differently.
Debugging is possible because the compiler inserts debugging information (such as function names, variable names, etc) into your executable. Its possible not to include this information.
Debuggers sometimes change the way the program behaves in order to work properly.
I'm not sure about Visual Studio but in Eclipse for example. Java classes are not loaded the same when ran inside the IDE and when ran outside of it.
You may also be having a race condition and the debugger stops one of the threads so when you continue the program flow it's at the right conditions.
More info on the program might help.
On Windows there is another difference caused by the debugger. When your program is launched by the debugger, Windows will use a different memory manager (heap manager to be exact) for your program. Instead of the default heap manager your program will now get the debug heap manager, which differs in the following points:
it initializes allocated memory to a pattern (0xCDCDCDCD comes to mind but I could be wrong)
it fills freed memory with another pattern
it overallocates heap allocations (like a previous answer mentioned)
All in all it changes the memory use patterns of your program so if you have a memory thrashing bug somewhere its behavior might change.
Two useful tricks:
Use PageHeap to catch memory accesses beyond the end of allocated blocks
Build using the /RTCsu (older Visual C++ compilers: /GX) switch. This will initialize the memory for all your local variables to a nonzero bit pattern and will also throw a runtime error when an unitialized local variable is accessed.