To debug a simulator which uses PIN tool, by using 2 different terminal windows, I do the following:
/home/agy/mcsim/pin/intel64/bin/pinbin -pause_tool 5 -t /home/agy/mcsim/Pthread/mypthreadtool -port 47145 -skip_first 10000 -- McSim/JacNoBoost/JACCARD
Pausing to attach to pid 5465
To load the tool's debug info to gdb use:
add-symbol-file /home/agy/mcsim/Pthread/mypthreadtool 0x7f86e6c06ee0 -s .data 0x7f86e709f620 -s .bss 0x7f86e70aaac0
On the other window I do the following:
gdb /home/agy/mcsim/Pthread/mypthreadtool
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/agy/mcsim/Pthread/mypthreadtool...done.
(gdb) attach 5465
Attaching to program: /home/agy/mcsim/Pthread/mypthreadtool, process 5465
0x0000000030592c70 in ?? ()
(gdb) add-symbol-file /home/agy/mcsim/Pthread/mypthreadtool 0x7f86e6c06ee0 -s .data 0x7f86e709f620 -s .bss 0x7f86e70aaac0
add symbol table from file "/home/agy/mcsim/Pthread/mypthreadtool" at
.text_addr = 0x7f86e6c06ee0
.data_addr = 0x7f86e709f620
.bss_addr = 0x7f86e70aaac0
(y or n) y
Reading symbols from /home/agy/mcsim/Pthread/mypthreadtool...done.
(gdb) c
Continuing.
At some point, pin tool freezes and i manually send interrupt signal to pin tool and try to debug it on the attached window. To understand at which line pin implementation stucks, I tried backtrace command but it returns with question marks even though the symbol file is added.
Program received signal SIGINT, Interrupt.
0x000000003052c474 in ?? ()
(gdb) backtrace
#0 0x000000003052c474 in ?? ()
#1 0x00007f86e5570850 in ?? ()
#2 0x000000003070c290 in ?? ()
#3 0x0000000001dda880 in ?? ()
#4 0x0000000001dda880 in ?? ()
#5 0x000000000040e03a in ?? ()
#6 0x0000000030643de0 in ?? ()
#7 0x00007f86d5ada0c0 in ?? ()
#8 0x0000000000000012 in ?? ()
#9 0x00007f86e5571350 in ?? ()
#10 0x0000000030aade20 in ?? ()
#11 0x0000000000000000 in ?? ()
I couldn't find the wrong approach here. Could you please help me?
I couldn't find the wrong approach here.
You are telling GDB that mypthreadtool is your main executable, but it is not: /home/agy/mcsim/pin/intel64/bin/pinbin is.
Try this:
gdb /home/agy/mcsim/pin/intel64/bin/pinbin 5465
(gdb) add-symbol-file /home/agy/mcsim/Pthread/mypthreadtool \
0x7f86e6c06ee0 -s .data 0x7f86e709f620 -s .bss 0x7f86e70aaac0
Related
I'm using the CrashCatcher/CrashDebug code from https://github.com/adamgreen/CrashDebug.
My platform is an STM32H753 running freertos. I've been able to generate a dump file based on the HexDump.c example in CrashCatcher. So far so good. The problem arises when I try to run arm-none-eabi-gdb.exe and connect it to CrashDebug.exe. I'm running on windows 10 and using the stm32 tool chain. I built CrashDebug.exe locally using mingw. I run the following command line:
C:\ST\STM32CubeIDE_1.6.1\STM32CubeIDE\plugins\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.9-2020-q2-update.win32_2.0.0.202105311346\tools\bin\arm-none-eabi-gdb.exe "fw-10143-sensor-hub-stm32-firmware\Debug\THOR_1.5_Sensor_Hub_FW.elf" -ex "set target-charset ASCII" -ex "target remote | /Users/felix/source/Thor/CrashDebug.exe --elf C:\Users\felix\source\Thor\fw-10143-sensor-hub-stm32-firmware\Debug\THOR_1.5_Sensor_Hub_FW.elf --dump C:\Users\felix\source\Thor\crash.dmp"
This runs gdb but the command interpreter becomes unresponsive and never returns to the gdb prompt. I can not even ctrl-c to break. I have to kill the process. Here is what gdb outputs before the hang.
C:\ST\STM32CubeIDE_1.6.1\STM32CubeIDE\plugins\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.9-2020-q2-update.win32_2.0.0.202105311346\tools\bin\arm-none-eabi-gdb.exe: warning: Couldn't determine a path for the index cache directory.
GNU gdb (GNU Tools for STM32 9-2020-q2-update.20201001-1621) 8.3.1.20191211-git
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-w64-mingw32 --target=arm-none-eabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from fw-10143-sensor-hub-stm32-firmware\Debug\THOR_1.5_Sensor_Hub_FW.elf...
Remote debugging using | /Users/felix/source/Thor/CrashDebug.exe --elf C:\Users\felix\source\Thor\fw-10143-sensor-hub-stm32-firmware\Debug\THOR_1.5_Sensor_Hub_FW.elf --dump C:\Users\felix\source\Thor\crash.dmp
I'd expect the gdb prompt to come back so I can get a stack trace out.
I tried running an instance of the mingw gdb and connecting to the running CrashDebug.exe process and halting it. When I do this I get the following stack trace.
(gdb) symbol-file c:/users/felix/source/Thor/CrashDebug/bins/win32/CrashDebug.exe
Load new symbol table from "c:\users\felix\source\Thor\CrashDebug\bins\win32\CrashDebug.exe"? (y or n) y
Reading symbols from c:\users\felix\source\Thor\CrashDebug\bins\win32\CrashDebug.exe...done.
(gdb) thread apply all bt
Thread 2 (Thread 3240.0x2804):
#0 0x77154d11 in ntdll!DbgBreakPoint () from C:\WINDOWS\SYSTEM32\ntdll.dll
#1 0x7718dca9 in ntdll!DbgUiRemoteBreakin () from C:\WINDOWS\SYSTEM32\ntdll.dll
#2 0xaa43fb20 in ?? ()
#3 0x7718dc70 in ntdll!DbgUiIssueRemoteBreakin () from C:\WINDOWS\SYSTEM32\ntdll.dll
#4 0x7556fa29 in KERNEL32!BaseThreadInitThunk () from C:\WINDOWS\System32\kernel32.dll
#5 0x77147a7e in ntdll!RtlGetAppContainerNamedObjectPath () from C:\WINDOWS\SYSTEM32\ntdll.dll
#6 0x77147a4e in ntdll!RtlGetAppContainerNamedObjectPath () from C:\WINDOWS\SYSTEM32\ntdll.dll
#7 0x00000000 in ?? ()
Thread 1 (Thread 3240.0xfc4):
#0 0x77152a1c in ntdll!ZwWriteFile () from C:\WINDOWS\SYSTEM32\ntdll.dll
#1 0x7509f32c in WriteFile () from C:\WINDOWS\System32\KernelBase.dll
#2 0x00000134 in ?? ()
#3 0x00000000 in ?? ()
(gdb)
Not very helpful. I'm at a loss as to what to do next. Because of the way CrashDebug is launched I can not breakpoint and step through the code to see what is going wrong. Can anyone advise me as to how to get this working?
This issue has been resolved (very quickly) by the awesome author.
https://github.com/adamgreen/CrashDebug/issues/18
When I run a program with gdb and have a commands list, the commands aren't invoked the first time the breakpoint is hit. I setup a little test program to demonstrate.
test.c:
#include <stdio.h>
void incrementI(){
static int i = 0;
i++;
printf("i: %d\n", i);
sleep(10);
}
int main() {
int i = 0;
while(1){
incrementI();
}
return 0;
}
commands list: testgdbBreakpoint
set breakpoint pending on
b incrementI
commands
backtrace
continue
end
shell script which runs gdb: testgdbinvoke.sh
#!/bin/sh
gdb -x testGDBBreakpoint -ex=r --args testGDB
The output I see when running testgdbinvoke.sh is:
GNU gdb (GDB) 7.12.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from testGDB...done.
Breakpoint 1 at 0x10478
Starting program: /data/config/testGDB
Breakpoint 1, 0x00010478 in incrementI ()
(gdb) c
Continuing.
i: 1
Breakpoint 1, 0x00010478 in incrementI ()
#0 0x00010478 in incrementI ()
#1 0x000104d8 in main ()
i: 2
Breakpoint 1, 0x00010478 in incrementI ()
#0 0x00010478 in incrementI ()
#1 0x000104d8 in main ()
i: 3
The first time the break point is hit, I don't see a backtrace, instead I have to manually continue.
Are there specific options I have to set in the commands list in order to have them run everytime the breakpoint is hit?
Reproduced using GDB-10.0.
There appears to be some kind of unexpected interaction between using multiple -x or -ex arguments. This doesn't work:
gdb -x testgdbBreakpoint -ex run ./a.out
gdb -ex 'source testgdbBreakpoint' -ex run ./a.out
But adding run at the end of testgdbBreakpoint works as expected:
gdb -q -x 'testgdbBreakpoint' ./a.out
Breakpoint 1 at 0x401136: file test.c, line 4.
Breakpoint 1, incrementI () at test.c:4
4 i++;
#0 incrementI () at test.c:4
#1 0x0000000000401189 in main () at test.c:11
i: 1
Breakpoint 1, incrementI () at test.c:4
4 i++;
#0 incrementI () at test.c:4
#1 0x0000000000401189 in main () at test.c:11
i: 2
^C
I downloaded FabariaGest source code then to compile it you have to run:
on qt 4 systems:
cmake -DMAKE_INSTALL_PREFIX=directory -DWANT_QT4=ON -DWANT_QWT=ON
on qt5 systems:
cmake -DMAKE_INSTALL_PREFIX=directory -DWANT_QT5=ON -DWANT_QWTQT5=ON
When it finishes, you run
# make install
Since I get a segmentation fault when I start the software, I tried to run
cd /opt/fabaria_gest
gdb fabaria_gest
run
backtrace
but I only get
[user#localhost fabaria_gest]$ gdb fabaria_gest
GNU gdb (GDB) Fedora 7.9.1-19.fc22
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from fabaria_gest...(no debugging symbols found)...done.
(gdb) run
Starting program: /opt/fabaria_gest/fabaria_gest
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Detaching after fork from child process 7246.
[New Thread 0x7fffe0798700 (LWP 7244)]
Program received signal SIGSEGV, Segmentation fault.
0x00000000004f568c in QBasicAtomicInteger<int>::load() const ()
(gdb) backtrace
#0 0x00000000004f568c in QBasicAtomicInteger<int>::load() const ()
#1 0x0000000000502a92 in QtPrivate::RefCount::ref() ()
#2 0x0000000000502e37 in QString::QString(QString const&) ()
#3 0x00000000006b9709 in _ZL13getSystemInfov ()
#4 0x00000000006bc29f in main ()
(gdb)
#0 0x00
What can I do to recompile the software in order to get better debuginfos? Setting set follow-fork-mode in GDB did not help
Using
cmake -DCMAKE_BUILD_TYPE=Debug ...
will instruct cmake to generate debug information, which should make the backtrace easier to read.
I'm using the Point Cloud Library with cmake for compilation, and I've got it building in debug mode, but my program doesn't seg fault or abort in the way I'd expect it to.
Specifically, I get messages like this:
(gdb) run bunny
Starting program: debug/our_cvfh bunny
libc++abi.dylib: terminating
[New Thread 0x170b of process 80178]
Program received signal SIGABRT, Aborted.
0x00007fff88c6f866 in ?? ()
(gdb) bt
#0 0x00007fff88c6f866 in ?? ()
#1 0x00007fff8bb5235c in ?? ()
#2 0x0000000000000000 in ?? ()
(gdb) break rec/registered_views_source.h:305
Cannot access memory at address 0x961d60
In this case, I know where the error is, but I'd like to be able to backtrace it and see what called the function in this case.
Is PCL creating another thread, and that's why I can't backtrace? I'm not doing any visualizations right now, so I can't figure out why it'd be using threading.
I've also tried running the program in the debug directory instead of from my source root directory. Here's another example of it not working:
$ gdb our_cvfh
GNU gdb (GDB) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin13.1.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from our_cvfh...done.
run (gdb) run
Starting program: /Users/jwoods/Projects/lidargl/fpfh/debug/our_cvfh
[New Thread 0x170b of process 33571]
Program received signal SIGSEGV, Segmentation fault.
0x000000010016cdec in ?? ()
(gdb) bt
#0 0x000000010016cdec in ?? ()
#1 0x00007fff5fbfbd08 in ?? ()
#2 0x00007fff5fbfbcc0 in ?? ()
#3 0x00007fff5fbfbcc8 in ?? ()
#4 0x00007fff5fbfbcc8 in ?? ()
#5 0x00007fff5fbfbcc8 in ?? ()
#6 0xffffffffffffffff in ?? ()
#7 0x00007fff5fbfbcc8 in ?? ()
#8 0x00007fff5fbfbcc8 in ?? ()
#9 0x00007fff5fbfbcc0 in ?? ()
#10 0x00007fff5fbfbcc0 in ?? ()
#11 0x00007fff5fbfbcc8 in ?? ()
#12 0x00007fff5fbfbcc8 in ?? ()
#13 0x00007fff5fbfbcc8 in ?? ()
#14 0x00007fff5fbff4a8 in ?? ()
#15 0x00007fff5fbff4d8 in ?? ()
#16 0x00007fff5fbff420 in ?? ()
#17 0x00007fff5fbff4d8 in ?? ()
#18 0x0000000000000000 in ?? ()
(gdb)
gdb works fine when I'm not using CMake, so my guess is it has something to do with CMake. This doesn't seem to present a problem for anyone else, which tells me it may also have to do with the fact that I'm using CMake with Mac OS X.
How do I get my normal GDB behavior?
Update
I can run dsymutil my_output_binary in order to generate the debugging symbols (following make). This is a workaround. I'd like it to be done automatically, and amn't sure why it's not. The dsymutil strategy works for most segfaults, but doesn't work for some cases of SIGABRT. Here is the output:
Calling compute <--- normal std::cerr output of my program, single-threaded
Assertion failed: (index >= 0 && index < size()), function operator[], file /usr/local/Cellar/eigen/3.2.1/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h, line 378.
[New Thread 0x170b of process 64108]
Program received signal SIGABRT, Aborted.
0x00007fff84999866 in ?? ()
(gdb) bt
#0 0x00007fff84999866 in ?? ()
#1 0x00007fff862c335c in ?? ()
#2 0x0000000000000000 in ?? ()
(gdb) info threads
Id Target Id Frame
2 Thread 0x170b of process 64108 0x00007fff8499a662 in ?? ()
* 1 Thread 0x1503 of process 64108 0x00007fff84999866 in ?? ()
(gdb)
Note that my program is not itself multi-threaded, but it appears to be making use of a library which is creating threads — or at least that's what I gather from the gdb output.
I've tried disabling threading with Eigen, which is used by PCL.
Interestingly, lldb is able to generate the backtraces, but I'm curious as to why GDB can't.
Other than simply using LLDB in lieu of GDB, I haven't figured out how to debug threads.
However, I have figured out how to automatically produce debugging symbols! Hooray.
You need three files:
UseCompVer.cmake
AddOptions.cmake
UseDebugSymbols.cmake
Put these in your project's cmake/Modules/ directory.
In CMakeLists.txt, you'll need the following lines after your project declaration:
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake/Modules/")
if (APPLE) # this if statement is optional, but you definitely need the include()
include(UseDebugSymbols)
endif (APPLE)
That will look in the appropriate location.
I should note that I made another change to my project simultaneously, which may also be useful to you. In my add_executable line, right after the name of the target, I added MACOSX_BUNDLE. This flag will cause it to be compiled as a .app instead of a regular binary. Here's an example from my project:
add_executable(pose MACOSX_BUNDLE pose.cpp rec/global_nn_recognizer_cvfh.cpp rec/global_nn_recognizer_cvfh.hpp rec/render_views_tesselated_sphere.cpp)
I'm trying to use GDB (v 7.3.1) from macports to debug an executable compiled with
g++ 4.7 (also from macports). However, I start the debugger, I get the following output:
GNU gdb (GDB) 7.3.1 Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://gnu.org/licenses/gpl.html This is free software: you are free
to change and redistribute it. There is NO WARRANTY, to the extent
permitted by law. Type "show copying" and "show warranty" for
details. This GDB was configured as "x86_64-apple-darwin11.2.0". For
bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/... BFD: unable to read unknown
load command 0x24 BFD: unable to read unknown load command 0x26 BFD:
unable to read unknown load command 0x24 BFD: unable to read unknown
load command 0x26
Then, whenever I attempt to backtrace, the debugger is unable to provide any
information about the stack; I get output like:
> #0 0x00000001085106d5 in ?? ()
> #1 0x000000010833d9b6 in ?? ()
> #2 0x0000000107e23000 in ?? ()
> #3 0x00007f828341f630 in ?? ()
> #4 0x00007f828341bc10 in ?? ()
> #5 0x00007f8283421950 in ?? ()
> #6 0x00007f8283427020 in ?? ()
> #7 0x00007f8283425510 in ?? ()
> #8 0x00007f82834195e0 in ?? ()
> #9 0x0000000100000001 in ?? ()
> #10 0x0000000100000000 in ?? ()
I've built with debug symbols, and I'm able to debug the executable fine on
linux. Does anyone know what's going on here or how I might fix it. It's
really difficult to fix certain problems without a debugger!
I don't know much about OSX or usage of gdb on it, but this appears to have been fixed shorly
after the release of gdb 7.3.1, I seem to recall something about the release of OSX and the release of gdb or binutils being around the same time, with the developer not having any early access to OS X. You could try a later version of gdb.
I'm guessing that the following patch committed shortly after 7.3.1 fixes it:
http://permalink.gmane.org/gmane.comp.gnu.binutils/54607
GDB 7.4.1 had fixed this issue, you could get and build it.
http://sourceware.org/gdb/wiki/BuildingOnDarwin