In my CMakeLists file, at the very end I have set(CMAKE_BUILD_TYPE Debug). However, when I try to do a backtrace, all I get is
#0 Car::setTexture (this=0x10025cc20, tex=...)
#1 0x0000000100002a0f in ?? ()
#2 0x0000000020000000 in ?? ()
#3 0x0000000020000000 in ?? ()
#4 0x0000000120000000 in ?? ()
#5 0x000000010023ce90 in ?? ()
#6 0x0000008000000060 in ?? ()
So far, everything else I've tried with debugging (list, etc) has worked fine. Only the backtraces are missing the function names. However, when I regenerate the build files with the cmake set(CMAKE_BUILD_TYPE Debug) commented out, the debugging symbols are missing (as expected), but the backtrace function calls are shown.
(gdb) backtrace
#0 0x00000001000056b4 in Car::setTexture(TRTexture) ()
#1 0x0000000100002a0f in SDLWrapper::loadSprite(Sprite*) ()
#2 0x0000000100004654 in TypeRight::startGame() ()
#3 0x0000000100001cc8 in main ()
However, when I regenerate the build files with the cmake set(CMAKE_BUILD_TYPE Debug) commented out, the debugging symbols are missing (as expected), but the backtrace function calls are shown.
The most likely cause: you are pointing GDB at the wrong executable.
As far as I understand, changing CMAKE_BUILD_TYPE changes the build output directory, and if you have not adjusted your GDB command for that, and if you are attaching to a running process rather than running from inside GDB, then that would explain all the symptoms you've observed.
Related
So I am experiencing this really weird behavior of gdb on Linux (KDE Neon 5.20.2):
I start gdb and load my executable using the file command:
(gdb) file mumble
Reading symbols from mumble...
As you can see it did find debug symbols. Then I start my program (using start) which causes gdb to pause at the entry to the main function. At this point I can also print out the back trace using bt and it works as expected.
If I now continue my program and interrupt it at any point during startup, I can still display the backtrace without issues. However if I do something in my application that happens in another thread than the startup (which all happens in thread 1) and interrupt my program there, gdb will no longer be able to display the stacktrace properly. Instead it gives
(gdb) bt
#0 0x00007ffff5bedaff in ?? ()
#1 0x0000555556a863f0 in ?? ()
#2 0x0000555556a863f0 in ?? ()
#3 0x0000000000000004 in ?? ()
#4 0x0000000100000001 in ?? ()
#5 0x00007fffec005000 in ?? ()
#6 0x00007ffff58a81ae in ?? ()
#7 0x0000000000000000 in ?? ()
which shows that it can't find the respective debug symbols.
I compiled my application with cmake (gcc) using -DCMAKE_BUILD_TYPE=Debug. I also ensured that a bunch of debug symbols are present in the binary using objdump --debug mumble (Which also printed a few lines of objdump: Error: LEB value too large, but I'm not sure if this is related to the problem I am seeing).
While playing around with gdb, I also encountered the error
Cannot find user-level thread for LWP <SomeNumber>: generic error
a few times, which lets me suspect that maybe there is indeed some issue invloving threads here...
Finally I tried starting gdb and before loading my binary using set verbose on which yields
(gdb) set verbose on
(gdb) file mumble
Reading symbols from mumble...
Reading in symbols for /home/user/Documents/Git/mumble/src/mumble/main.cpp...done.
This does also look suspicious to me as only main.cpp is explicitly listed here (even though the project has much, much more source files). I should also note that all successful backtraces that I am able to produce (as described above) all originate from main.cpp.
I am honestly a bit clueless as to what might be the root issue here. Does someone have an idea what could be going on? Or how I could investigate further?
Note: I also tried using clang as a compiler but the result was the same.
Used program versions:
cmake: 3.18.4
gcc: 9.3.0
clang: 10.0.0
make: 4.2.1
I am trying to debug my c++ programming assignment application using gdb on an Ubuntu server, because it produces segmentation fault.
But the file produces ?? symbols that are unreadable to me when I try bt it gives me.
(gdb) bt
#0 0x00007f141956d277 in ?? ()
#1 0x00007ffc1a866bd0 in ?? ()
#2 0x000055e1f101d5e0 in ?? ()
#3 0x00007ffc1a866db0 in ?? ()
#4 0x000055e1f1433e70 in ?? ()
#5 0x00007ffc1a866bd0 in ?? ()
#6 0x000055e1f10224a9 in ?? ()
#7 0x000055e1f14341f8 in ?? ()
#8 0x00000001f14344d0 in ?? ()
#9 0x0000000000000000 in ?? ()
I was following this link, and it told me to load these symbols
symbol-file /path/to/my/binary
sharedlibrary
The sharedlibrary was found, but the symbol-file path is not there. So,it did change bt command output somehow
(gdb) bt
#0 tcache_get (tc_idx=0) at malloc.c:2943
#1 _GI__libc_malloc (bytes=19) at malloc.c:3050
#2 0x000055e1f10224a9 in ?? ()
#3 0x000055e1f14341f8 in ?? ()
#4 0x00000001f14344d0 in ?? ()
#5 0x0000000000000000 in ?? ()
I still don't understand the bug.
Now, I don't know it's a problem from the GDB for not having this symbol-file or its a compilation problem which I don't know how or that's enough for me to debug, but I was following Debugging a Segmentation Fault and it was much clearer to troubleshoot.
When I search for similar cases, all of them were answered only for their case, not a general solution how to deal with these kinds of error. I also thought of installing or locating that symbol-file but I didn't understand how.
If someone could help me, I need to understand what is my problem and how should I fix it.
Note: core dump is produced in the /tmp not in current application directory
I was following this link, and it told me to load these symbols
Don't follow this link (it's unnecessarily complicated for your use case).
Instead, do this:
gdb /path/to/my/binary
(gdb) run
... GDB will stop when your program encounters SIGSEGV
(gdb) bt # should produce meaningful output now.
It seems that gdb fails finding the code position of an assertion failure, after I recompile my code. More precisely, I expect the position of a signal raise, relative to an assertion failure, to be
0x00007ffff7a5ff00 in raise () from /lib64/libc.so.`6
while instead I obtain
0x00007ffff7a5ff00 in ?? ()
For instance, consider the following code
#include <assert.h>
int main()
{
assert(0);
return 0;
}
compiled with debug symbols and debugged with gdb.
> gcc -g main.c
> gdb a.out
On the first run of gdb, the position is found, and the backtrace is reported correctly:
GNU gdb (Gentoo 8.0.1 p1) 8.0.1
...
(gdb) r
Starting program: /home/myself/a.out
a.out: main.c:5: main: Assertion `0' failed.
Program received signal SIGABRT, Aborted.
0x00007ffff7a5ff00 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x00007ffff7a5ff00 in raise () from /lib64/libc.so.6
#1 0x00007ffff7a61baa in abort () from /lib64/libc.so.6
#2 0x00007ffff7a57cb7 in ?? () from /lib64/libc.so.6
#3 0x00007ffff7a57d72 in __assert_fail () from /lib64/libc.so.6
#4 0x00005555555546b3 in main () at main.c:5
(gdb)
The problem comes when I recompile the code. After recompiling, I issue the run command in the same gdb instance. Gdb re-reads the symbols, starts the program from the beginning, but does not find the right position:
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
`/home/myself/a.out' has changed; re-reading symbols.
Starting program: /home/myself/a.out
a.out: main.c:5: main: Assertion `0' failed.
Program received signal SIGABRT, Aborted.
0x00007ffff7a5ff00 in ?? ()
(gdb) bt
#0 0x00007ffff7a5ff00 in ?? ()
#1 0x0000000000000000 in ?? ()
(gdb) up
Initial frame selected; you cannot go up.
(gdb) n
Cannot find bounds of current function
At this point the debugger is unusable. One cannot go up, step forward.
As a workaround, I can manually reload the file, and positions are found again.
(gdb) file a.out
Load new symbol table from "a.out"? (y or n) y
Reading symbols from a.out...done.
(gdb) r
Starting program: /home/myself/a.out
a.out: main.c:5: main: Assertion `0' failed.
Program received signal SIGABRT, Aborted.
0x00007ffff7a5ff00 in raise () from /lib64/libc.so.6
(gdb)
Unfortunately, after reloading the file this way, gdb fails resetting the breakpoints.
ERRATA CORRIGE: I was experiencing failure in resetting the breakpoints using gdb 7.12.1. After upgrading to 8.0.1 the problem vanished. Supposedly, this was related to the bugfix https://sourceware.org/bugzilla/show_bug.cgi?id=21555. However, code positions where assertions fail still cannot be found correctly.
Does anybody have any idea about what is going on here?
This has started happening after a system update. The system update recompiled all system libraries, including the glibc, as position independent code, i.e., compiled with -fPIC.
Also, the version of the gcc I am using is 6.4.0
Here is a workaround. Since file re-reads the symbols correctly, while run does not, we can define a hook for the command run so to execute file before:
define hook-run
pi gdb.execute("file %s" % gdb.current_progspace().filename)
end
after you change the source file and recompile u are generating a different file from the one loaded to GDB.
you need to stop the running debug cession and reload the file.
you cant save the previously defined breakpoints and watch points in the file to a changed source, since gdb is actually inserting additional code to your source to support breakpoints and registrar handlers.
if you change the source the the behavior is undefined and you need to reset those breakpoints.
you can refer to gdb manual regarding saving breakpoints in a file as
Mark Plotnick suggested, but it wont work if you change the file(from my experience)
https://sourceware.org/gdb/onlinedocs/gdb/Save-Breakpoints.html
I'm using the Point Cloud Library with cmake for compilation, and I've got it building in debug mode, but my program doesn't seg fault or abort in the way I'd expect it to.
Specifically, I get messages like this:
(gdb) run bunny
Starting program: debug/our_cvfh bunny
libc++abi.dylib: terminating
[New Thread 0x170b of process 80178]
Program received signal SIGABRT, Aborted.
0x00007fff88c6f866 in ?? ()
(gdb) bt
#0 0x00007fff88c6f866 in ?? ()
#1 0x00007fff8bb5235c in ?? ()
#2 0x0000000000000000 in ?? ()
(gdb) break rec/registered_views_source.h:305
Cannot access memory at address 0x961d60
In this case, I know where the error is, but I'd like to be able to backtrace it and see what called the function in this case.
Is PCL creating another thread, and that's why I can't backtrace? I'm not doing any visualizations right now, so I can't figure out why it'd be using threading.
I've also tried running the program in the debug directory instead of from my source root directory. Here's another example of it not working:
$ gdb our_cvfh
GNU gdb (GDB) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin13.1.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from our_cvfh...done.
run (gdb) run
Starting program: /Users/jwoods/Projects/lidargl/fpfh/debug/our_cvfh
[New Thread 0x170b of process 33571]
Program received signal SIGSEGV, Segmentation fault.
0x000000010016cdec in ?? ()
(gdb) bt
#0 0x000000010016cdec in ?? ()
#1 0x00007fff5fbfbd08 in ?? ()
#2 0x00007fff5fbfbcc0 in ?? ()
#3 0x00007fff5fbfbcc8 in ?? ()
#4 0x00007fff5fbfbcc8 in ?? ()
#5 0x00007fff5fbfbcc8 in ?? ()
#6 0xffffffffffffffff in ?? ()
#7 0x00007fff5fbfbcc8 in ?? ()
#8 0x00007fff5fbfbcc8 in ?? ()
#9 0x00007fff5fbfbcc0 in ?? ()
#10 0x00007fff5fbfbcc0 in ?? ()
#11 0x00007fff5fbfbcc8 in ?? ()
#12 0x00007fff5fbfbcc8 in ?? ()
#13 0x00007fff5fbfbcc8 in ?? ()
#14 0x00007fff5fbff4a8 in ?? ()
#15 0x00007fff5fbff4d8 in ?? ()
#16 0x00007fff5fbff420 in ?? ()
#17 0x00007fff5fbff4d8 in ?? ()
#18 0x0000000000000000 in ?? ()
(gdb)
gdb works fine when I'm not using CMake, so my guess is it has something to do with CMake. This doesn't seem to present a problem for anyone else, which tells me it may also have to do with the fact that I'm using CMake with Mac OS X.
How do I get my normal GDB behavior?
Update
I can run dsymutil my_output_binary in order to generate the debugging symbols (following make). This is a workaround. I'd like it to be done automatically, and amn't sure why it's not. The dsymutil strategy works for most segfaults, but doesn't work for some cases of SIGABRT. Here is the output:
Calling compute <--- normal std::cerr output of my program, single-threaded
Assertion failed: (index >= 0 && index < size()), function operator[], file /usr/local/Cellar/eigen/3.2.1/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h, line 378.
[New Thread 0x170b of process 64108]
Program received signal SIGABRT, Aborted.
0x00007fff84999866 in ?? ()
(gdb) bt
#0 0x00007fff84999866 in ?? ()
#1 0x00007fff862c335c in ?? ()
#2 0x0000000000000000 in ?? ()
(gdb) info threads
Id Target Id Frame
2 Thread 0x170b of process 64108 0x00007fff8499a662 in ?? ()
* 1 Thread 0x1503 of process 64108 0x00007fff84999866 in ?? ()
(gdb)
Note that my program is not itself multi-threaded, but it appears to be making use of a library which is creating threads — or at least that's what I gather from the gdb output.
I've tried disabling threading with Eigen, which is used by PCL.
Interestingly, lldb is able to generate the backtraces, but I'm curious as to why GDB can't.
Other than simply using LLDB in lieu of GDB, I haven't figured out how to debug threads.
However, I have figured out how to automatically produce debugging symbols! Hooray.
You need three files:
UseCompVer.cmake
AddOptions.cmake
UseDebugSymbols.cmake
Put these in your project's cmake/Modules/ directory.
In CMakeLists.txt, you'll need the following lines after your project declaration:
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake/Modules/")
if (APPLE) # this if statement is optional, but you definitely need the include()
include(UseDebugSymbols)
endif (APPLE)
That will look in the appropriate location.
I should note that I made another change to my project simultaneously, which may also be useful to you. In my add_executable line, right after the name of the target, I added MACOSX_BUNDLE. This flag will cause it to be compiled as a .app instead of a regular binary. Here's an example from my project:
add_executable(pose MACOSX_BUNDLE pose.cpp rec/global_nn_recognizer_cvfh.cpp rec/global_nn_recognizer_cvfh.hpp rec/render_views_tesselated_sphere.cpp)
C++ application gets crashed with the core file showing error
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff79e54000
Core was generated by `./server'.
Program terminated with signal 6, Aborted.
#0 0x0000003b67230265 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x0000003b67230265 in raise () from /lib64/libc.so.6
#1 0x0000003b67231d10 in abort () from /lib64/libc.so.6
#2 0x0000003b6726a9bb in __libc_message () from /lib64/libc.so.6
#3 0x0000003b6727247f in _int_free () from /lib64/libc.so.6
#4 0x0000003b672728db in free () from /lib64/libc.so.6
#5 0x00000000004060df in operator delete (p=0x20030190) at ../lib/m_string.cpp:43
#6 0x0000000000403892 in TStr::~TStr (this=0x2102c980, __in_chrg=<value optimized out>) at ../lib/m_string.cpp:175 –
could able to understand about this issue. Here is the link that i have verified https://bugzilla.redhat.com/show_bug.cgi?id=959013
shows that the size of vdso file is not enough. which is in path /proc/self/maps.
please let me know what kind of issue is this and please suggest a fix for this.
what kind of issue is this
Any crash inside malloc or free is a sure sign of previous heap corruption.
Use Valgrind or AddressSanitizer (incorporated into GCC-4.8 as well) to find the root cause.
Ignore vdso -- as Tom Tromey said, it has nothing to do with the problem.