What are the 'best practices' when it comes to debugging core dumps using GDB?
Currently, I am facing a problem:
The release version of my application is compiled without the '-g' compiler flag.
The debug version of my application (compiled with '-g') is archived (along with the source code, and a copy of the release binary).
Recently, when a user gave me a core dump, I tried debugging it using
gdb --core=./core.pid ./my_app_debug-bin
The core was created by my_app_release-bin. There seems to be some kind of mismatch between the core file and the binary.
On the other hand, if I try
gdb --core=./core.pid ./my_app_release-bin
the core matches but I am unable to get source code line numbers (although I get the function names).
Is this what is practised? Because I feel I am missing something here.
It sounds like there are other differences between your release and debug build then simply the absence/presence of the -g flag. Assuming that's the case, there is nothing you can do right now, but you can adjust your build to handle this better:
Here's what we do at my place of work.
Include the -g flag when building the release version.
Archive that version.
run strip --strip-unneeded on the binary before shipping it to customers.
Now, when we get a crash we can use the archived version with symbols to do debugging.
One thing to note is that if your release version includes optimization, debugging may be difficult even with symbols. For example, the optimizer can reorder your code so even though the debugger will say you crashed on line N, you can't assume that the code actually executed line N-1.
You need to do some additional stuff to create binaries with stripped debug information that you can then debug from cores. Best description I could find is here
No, you don't miss anything. debug and release are just different binaries, so the core files of release don't match the debug binary. You have to look at machine code to get something from the release core dump.
You probably have to ask your user how the crash happened and collect additional log information or whatever you app produces.
Related
I have problem. Everytime when I compile in release build with cmake (using CLion), it also creates DWARF debug information, although it is release build. When I open my exe in IDA-Pro, it is really easy to reverse-engineer (even though I have no skills in reverse-engineering whatsoever). I tried setting the O3 flag, I tried using -g0, but nothing works. The debug information (DWARF) is still included in my exe, no matter what I do. The only fix I found is using llvm-strip, but I don't want to do that all the time.
When I build in Debug mode, the executable is only 4kb bigger, it has 126 kb in release, 130 in debug. And it has 46kb with llvm-strip --strip-debug. So the release build obviously contains the debug informations. Is there anyway for me to delete it with flag instead of using separate tool?
I am writing a program using OSG and I would like to step through it to see how stuff works, change a few lines and debug again. However every time I need to load a file using osgDB plugin for openflight. The debug version of the plugin works quite slow. I have no intention of debugging the plugin itself, so I figured I might as well use the release dll. I read that I can do this as long as there are no memory allocation in program and freeing in dll or visa versa and if the dll was designed to avoid problems that might occur with release/debug mixing. I'm not sure if osgdb_openflight.dll was designed like that.
My question: can I use the release plugin dll in my debug build, and how can I make my program use it?
UPD: I achieved my goal following the #XenonofArcticus's advice.
Some additional thoughts on the topic for the sake of better understanding:
I also tried to simply rename the release dll, so it would get loaded in the debug build. (I thought this would work, because I usually use release dlls in debug builds - Windows's, SQL's and other 3rd party dlls, that do not come with debug info. Also see Igor Tandetnik's comments on this question). It did and the program didn't crash, however it also didn't load the file. Apparently this particular dll wasn't designed to be used in debug builds. Any other explanation?
I'm not sure you're going to achieve what you want trying to mix debug and release. However, perhaps it would work to build a release with debug symbols target? This can allow you to do SOME debugging, but it's still a release build and behaves like one.
I created a simple cpp file and compiled it using the cygwin g++ compiler in Win7. I am now trying to debug the resulting executable in gdb, but I can't get it to behave the way I expect it to. I cannot place breakpoints because when I try to execute b file.cpp:25 I get back
No source file named file.cpp.
Make breakpoint pending on future shared library load? (y or [n])
I select y and it still does not break at the expected point. I did compile from this source.
I am getting a segfault at a certain point and whe also does not actually show line numbers. It seems to show memory addresses, which is obviously not useful to me.
Is gdb is misbehaving or am i just expecting it to do things it can't do? If it doesn't have this capability (though I've done this kind of thing before), is there another tool I can use?
In order to add debug information during compilation you should use the -g flag for g++.
I have a core dump generated by 2 applications -> /usr/bin/python and /usr/bin/app1.
I know the dump can be analyzed by
gdb /path/to/app /path/to/core
but is there a way to include both applications in the arguement?
I did try gdb '/usr/bin/python /usr/bin/app1' core.xxx but that doesnt seem right.
Any suggestions?
I think you cannot achieve what you want with a single invocation of gdb. But you could run gdb twice, in different terminal windows. I did that more than once, and it works quite well (except of course that your own brain could be slightly overloaded).
a gdb process can debug only one single program, with one single debugged process or (for post mortem debug) one single core file.
And a given core file is produced by abnormal termination of one single process (not several), so I don't understand your question.
Apparently, you have a crash in some execution of python probably augmented by your faulty C code. I suggest having a debuggable variant of Python, perhaps by installing the python3-all-dbg package or something similar, then use gdb on it. Of course, compile your C code plugged into Python with debugging enabled. Perhaps you violated some invariant of the Python garbage collector.
I'm trying to find a way to debug core files sent to me from released versions of my software (c++ code compiled with gcc). Ideally, I'd like to be able to deploy release builds, and keep debug builds on hand to use for debugging, so I have symbol tables, etc.
My problem is that (as I understand it) the debug and release builds are not guaranteed to be the same - so a core file from the field may just look like garbage when I fire up gdb and point to my debug executable.
Is there a way around this (and here's the catch) without impacting size or performance of my released software? It's a large application, and performance of the debug build is probably not acceptable to customers. I've looked at suggestions to build once (debug), then strip symbol tables and ship that as the release build, but I'm going to see a performance hit with that approach, won't I?
Does anyone have suggestions for things they've tried or currently use that address this problem?
Thanks!
You can compile and link with optimization turned on and still generate debug symbols (-O3 -g) and then extract the debug symbols. This way you'd have the debug symbols around but can ship without them, and you won't have a performance penalty or something. See How to generate gcc debug symbol outside the build target? on how to do that.