gdb remote debugging: symbol addresses in local file not relocated - gdb

The following workflow worked for me at one point, and then it suddenly stopped working. I'd like to figure out why and get it to work again.
on the host, build binary for target system with debug information
send the stripped version of the binary to the target system
on the target, run the binary with gdbserver :6006 mybinary args...
on the host, run gdb-multiarch mybinary. This is the unstripped version. It shows that it loaded symbols from the local binary.
At the gdb prompt, run target remote <ip_of_target>:6006
Then, if I say b main, it inserts a breakpoint at an unrelocated address like 0x621730, which is the offset of the main function in the local binary, whereas it should be added on top of the VM address the remote binary is loaded at (0x5555555000 in this case.) Obviously the unrelocated address doesn't work.
So step 6 was working at one point. I don't know what I'm doing now differently to make the relocation no longer work. Help would be appreciated.
Version of gdbserver and that of host gdb are both Ubuntu 8.1.1-0ubuntu1. The remote system is aarch64.

So step 6 was working at one point. I don't know what I'm doing now differently to make the relocation no longer work.
I suspect that you have upgraded your toolchain, and that your upgraded toolchain produces a Position-Independent Executable mybinary by default, where the old toolchain didn't.
Try adding -no-pie to the link line of mybinary.
Obviously the unrelocated address doesn't work.
Actually, this does work for local debugging -- GDB is smart enough to realize that the code got loaded at a different address:
gdb -q a.out
Reading symbols from a.out...
(gdb) b main
Breakpoint 1 at 0x112d: file t.c, line 1. <<< Note: unrelocated address
(gdb) run
Starting program: /tmp/a.out
Breakpoint 1, main () at t.c:1
1 int main() {return 0; }
(gdb) p/x $pc
$1 = 0x55555555512d <<< Note: breakpoint worked.
It's possible that remote debugging doesn't work, although this worked fine for me using GDB-10.0:
strip a.out -o b.out
gdbserver :6006 ./b.out
Process ./b.out created; pid = 239653
Listening on port 6006
... in another window:
gdb -q ./a.out
Reading symbols from ./a.out...
(gdb) target remote :6006
Remote debugging using :6006
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading symbols from target:/lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/.build-id/a8/97a1105e21dd270bd418fe58c441700a6d8ec5.debug...
0x00007ffff7fe4940 in _start () from target:/lib64/ld-linux-x86-64.so.2
(gdb) b main
Breakpoint 1 at 0x55555555512d: file t.c, line 1.

Related

gdb remote debugging how to load symbols from remote [duplicate]

I am trying to use gdbserver...
I have an application with binary path /user/bin/foo running with pid 19767.
Started the gdbserver on remote:
gdbserver --remote-debug --multi 0:12347
Started gdb on client and connected it to remove server
target extended-remote 192.168.1.84:12347
Attached gdb on pid
attach 19767
It shows:
warning: Could not load vsyscall page because no executable was specified
try using the "file" command first.
0x00007f207550043c in ?? ()
Also, current thread information it is showing is incorrect. Like info threads shows 1 thread , but my app has 10 threads-
(gdb) info threads
* 1 Thread 19767.19767 0x00007f207550043c in ?? ()
How can I ask gdb to load symbol from remote file /user/bin/foo? How to make it show correct info?
How can I ask gdb to load symbol from remote file /user/bin/foo
You can't. Copy remote /usr/bin/foo locally (or mount the filesystem it's on), and then invoke gdb like this: gdb /path/to/copy/of/foo, or just use the file command.
As of gdb 7.10, you can use
set sysroot target:
to make gdb retrieve files from the remote filesystem. See https://sourceware.org/gdb/onlinedocs/gdb/Files.html#Files

GDB loading a different file when debuggin raspberry pi PICO?

I'm trying to debug a Raspberry Pi Pico project using openOCD + Picoprobe. I go through the steps that appear to be correct to do all this from my Linux PC (the Get started with Pico and C++ guide is for the Raspberry Pi, but there are tutorials online for PC), but when trying to set breakpoints in gdb, it appears to be looking at a different file.
My steps are the following:
In a console, navigate to the openOCD folder and run:
sudo src/openocd -s tcl/ -f tcl/interface/picoprobe.cfg -f tcl/target/rp2040.cfg
I get only info messages, no errors, until it waits for the gdb to connect
Info : starting gdb server for rp2040.core0 on 3333
Info : Listening on port 3333 for gdb connections
Then, in a different console, I navigate to what I want to debug, in this case the blink.c example. I stand inside the build folder with my .elf file and run
sudo gdb-multiarch blink.elf
(gdb) target extended-remote localhost:3333
(gdb) monitor reset init
(gdb) break 15
where 15 is a relevant debuggable line. But the thing is, the breakpoints are said to be set in a different file:
Breakpoint 1 at 0x20000178: file ../../../../../../libgcc/config/arm/lib1funcs.S, line 1461
and then on, all breakpoints are also set there. Here's an example trying to set 3 different breakpoints:
(gdb) b 15
Breakpoint 1 at 0x20000178: file ../../../../../../libgcc/config/arm/lib1funcs.S, line 1461.
(gdb) b 16
Note: breakpoint 1 also set at pc 0x20000178.
Breakpoint 2 at 0x20000178: file ../../../../../../libgcc/config/arm/lib1funcs.S, line 1461.
(gdb) b 17
Note: breakpoints 1 and 2 also set at pc 0x20000178.
Breakpoint 3 at 0x20000178: file ../../../../../../libgcc/config/arm/lib1funcs.S, line 1461.
So that. I am totally lost on what's happening here. Could anyone point me to why this is happening, or what that file does?
I see it's been a few months but I just ran into the same problem, here's what I've learned.
When you compile the C/C++ program that you're trying to debug, you need to compile it in debug mode, which tells the compiler to include debug symbols in the program. If compiling directly with gcc, you can use the -g flag. However, I'm assuming you're using CMake, in which case you can just add set(CMAKE_BUILD_TYPE Debug) to your CMakeLists.txt (found a few other options here as well)
Probably goes without saying, but after recompiling this way, you'll need to re-load your new program onto your target Pico by holding the bootsel button and plugging it into your machine directly (ie. not via your debugger Pico).
I also found that after doing this, setting breakpoints just to specific lines still didn't behave as expected, and I had to actually specify the name of the file as well, ex: break main.c:15

gdb list error "No such file or directory"

I'm currently running a file manager program that abruptly crashed with a segmentation fault and dumped a core file. So I used gdb to debug the core file as:
gdb /path/to/executable /path/to/core
The program which I was running is written in C++. When I ran GDB and tried to print the source lines using "list", I got the following error:
(gdb) bt
#0 0x0000000000554286 in
MyFSEventManager::AddEvent(wxFileSystemWatcherEvent&) ()
#1 0x00000000005ab2e8 in
MyGenericDirCtrl::OnFileWatcherEvent(wxFileSystemWatcherEvent&) ()
(gdb) f 0
#0 0x0000000000554286 in
MyFSEventManager::AddEvent(wxFileSystemWatcherEvent&) ()
(gdb) l
1 /build/glib2.0-prJhLS/glib2.0-2.48.2/./glib/gmain.c: No such file or directory.
Why does gdb say this "/build/glib2.0-prJhLS/glib2.0-2.48.2/./glib/gmain.c: No such file or directory." I do not hit this issue with some other programs that I've debugged using gdb.
The operating system used is Ubuntu 16.04 running on Oracle virtual box. I think may be the gdb symbols were not loaded. I'm not sure why since I compiled the program using the "-g" option. I really need to know the source lines where the code crashes via gdb.
Any suggestions?
EDIT: changes after suggestions from Employed Russian
I was compiling my main using "-g" option and linking it to "existing" object files which were obviously not compiled using "-g" so when the core dumped, I could not see the source for these files. So I went ahead and recompiled those files with "-g" option and reproduced the core dump. It's able to show me the source lines now.
Why does gdb say this "/build/glib2.0-prJhLS/glib2.0-2.48.2/./glib/gmain.c: No such file or directory."
Because you really don't have that file on your system.
I think may be the gdb symbols were not loaded
GDB did load debug symbols for glib, but not for your main executable.
I'm not sure why since I compiled the program using the "-g" option.
Since we don't have your compile and link lines, we can't tell exactly what's wrong, but some of the common issues are:
You have a "stray" -s or -Wl,-s on your link line (this strips debug info from the resulting binary).
You have -g when compiling your main.c, but not when compiling the source in which MyFSEventManager::AddEvent() is defined
P.S.
(gdb) bt
This doesn't seem to be the complete output from bt command. Always try to paste complete outputs as it makes helping easier :)

gdb/solaris: When attaching to a process, symbols not being loaded

I'm using gcc 4.9.2 & gdb 7.2 in Solaris 10 on sparc. The following was tested after compiling/linking with -g, -ggdb, and -ggdb3.
When I attach to a process:
~ gdb
/snip/
(gdb) attach pid_goes_here
... it is not loading symbolic information. I started with netbeans which starts gdb without specifying the executable name until after the attach occurs, but I've eliminated netbeans as the cause.
I can force it to load the symbol table under netbeans if I do one of the following:
Attach to the process, then in the debugger console do one of the following:
(gdb) detach
(gdb) file /path/to/file
(gdb) attach the_pid_goes_here
or
(gdb) file /path/to/file
(gdb) sharedlibrary .
I want to know if there's a more automatic way I can force this behavior. So far googling has turned up zilch.
I want to know if there's a more automatic way I can force this behavior.
It looks like a bug.
Are you sure that the main executable symbols are loaded? This bug says that attach pid without giving the binary doesn't work on Solaris at all.
In any case, it's supposed to work automatically, so your best bet to make it work better is probably to file a bug, and wait for it to be fixed (or send a patch to fix it yourself :-)

Using gdb and gdbserver with a 32 bit binary on a 64 bit machine with Centos 5 complains about memory access or badly formatted data

I have two identical 64 bit Centos 5 machines, that are networked, and share their /home mount. I compile a simple Hello World program on one, and then I have figured out how to use gdb on one machine to remotely debug it running on the other machine. That seems to work fine when everyone defaults to 64 bitness.
However, if I compile my Hello World with -m32 to generate a 32 bit binary, the way our full up system is being compiled, then I cannot figure out how to get gdb and gdbserver to correctly connect. Before I try it on our full up system, I figure I should get it working with hello. Depending on how I try connecting gdb and gdbserver, I either get messages about badly formatted registers, warnings about architecture mismatches, or illegal memory references.
I seem to have little understanding of what the implications of -m32 are in my compile and no idea of how to start gdb and gdbserver or the right order to specify architecture or files or something. :(
What does it take to use gdb and gdbserver on a 32 bit (-m32) executable on a 64 bit linux box?
Examples below, and thank you,
Jerry
hello.cpp:
#include <iostream>
int main(int argc, char *argv[])
{
std::cout << "Hello World." << std::endl;
return -1;
}
Here are three runs:
In gdb, set architecture i386 / then connect to gdbserver => bad architecture
In gdb, set architecture i386 / file hello / then connect to gdbserver => bad architecture
In gdb, set architecture (incorrectly) i386:x86-64 / file hello / then connect to gdbserver => Cannot access memory
Or in a bit more detail:
==============================
For each run, the remote gdbserver said:
$ gdbserver --multi rdev6:2010 hello
Process hello created; pid = 32603
Listening on port 2010
Remote debugging from host 134.51.26.149
readchar: Got EOF
Remote side has terminated connection. GDBserver will reopen the connection.
Listening on port 2010
And on our local:
==============================
Assuming it is i386 32 bit, setting archi to i386, then connecting
note: on the gdb side, the executable hasn't been specified or loaded
$ gdb
GNU gdb Fedora (6.8-37.el5)
his GDB was configured as "x86_64-redhat-linux-gnu".
(gdb) set archi i386
The target architecture is assumed to be i386
(gdb) target extended-remote rdev6:2010
Remote debugging using rdev6:2010
warning: Selected architecture i386 is not compatible with reported target architecture i386:x86-64
Remote register badly formatted: T0506:0000000000000000;07:b0dcdfff00000000;10:1018620000000000;thread:7f5b;
here: 0000000;07:b0dcdfff00000000;10:1018620000000000;thread:7f5b;
Try to load the executable by `file' first,
you may also check `set/show architecture'.
(gdb)
==============================
Assuming it is i386 32 bit, setting archi to i386, then connecting
note: on the gdb side, the executable has been loaded with file
$ gdb
GNU gdb Fedora (6.8-37.el5)
his GDB was configured as "x86_64-redhat-linux-gnu".
(gdb) set archi i386
The target architecture is assumed to be i386
(gdb) file hello
Reading symbols from /home/j/hello...done.
(gdb) target extended-remote rdev6:2010
Remote debugging using rdev6:2010
warning: Selected architecture i386 is not compatible with reported target architecture i386:x86-64
Remote register badly formatted: T0506:0000000000000000;07:b0dcdfff00000000;10:1018620000000000;thread:7f5b;
here: 0000000;07:b0dcdfff00000000;10:1018620000000000;thread:7f5b;
Try to load the executable by `file' first,
you may also check `set/show architecture'.
(gdb) sho archi
The target architecture is assumed to be i386
(gdb)
==============================
Assuming (which should be incorrect) that it is i386:x86-64, setting archi to i386:x86-64, then connecting
note: on the gdb side, the executable has been loaded with file
$ gdb
GNU gdb Fedora (6.8-37.el5)
This GDB was configured as "x86_64-redhat-linux-gnu".
(gdb) set archi i386:x86-64
The target architecture is assumed to be i386:x86-64
(gdb) file hello
Reading symbols from /home/j/hello...done.
(gdb) show archi
The target architecture is assumed to be i386:x86-64
(gdb) target extended-remote rdev6:2010
Remote debugging using rdev6:2010
[New Thread 32667]
Cannot access memory at address 0x800000008
(gdb)
If you want to debug 32-bit process using 64-bit gdb/gdbserver, you need a newer version of GDB. In particular, you need this:
gdbserver/ChangeLog:
2009-05-12 Doug Evans <dje#google.com>
Biarch support for i386/amd64 gdbserver.
Alternatively, you can build gdb/gdbserver you already have from source in 32-bit mode by running
./configure CC='gcc -m32'
and use gdb32/gdbserver32 to debug your processes. I don't see any advantage of doing this though -- newer versions of GDB have many fixes, speedups, and STL pretty printers are nice.