How to GDB step debug a dynamically linked executable in QEMU user mode? - gdb

Question

For example for ARM, if I compile statically, all works fine:
sudo apt-get install gdb-multiarch gcc-arm-linux-gnueabihf qemu-user
printf '
#include <stdio.h>
#include <stdlib.h>
int main() {
puts("hello world");
return EXIT_SUCCESS;
}
' > hello_world.c
arm-linux-gnueabihf-gcc -ggdb3 -static -o hello_world hello_world.c
qemu-arm -L /usr/arm-linux-gnueabihf -g 1234 ./hello_world
On another terminal:
gdb-multiarch -q --nh \
-ex 'set architecture arm' \
-ex 'set sysroot /usr/arm-linux-gnueabihf' \
-ex 'file hello_world' \
-ex 'target remote localhost:1234' \
-ex 'break main' \
-ex continue \
;
This leaves me at main, and I can see the source and step debug as usual.
However, if I remove the -static, and keep everything else unchanged, my breakpoint never gets hit, and the program runs until completion:
The target architecture is assumed to be arm
Reading symbols from hello_world...done.
Remote debugging using localhost:1234
Reading symbols from /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3...(no debugging symbols found)...done.
0xff7b3b80 in ?? () from /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3
Breakpoint 1 at 0x50c: file hello_world.c, line 5.
Continuing.
[Inferior 1 (Remote target) exited normally]
The executable itself does work fine however:
qemu-arm -L /usr/arm-linux-gnueabihf ./hello_world
prints:
hello world
I have seen: How to single step ARM assembler in GDB on Qemu? but it didn't cover the case of dynamically linked executables specifically.
Tested on Ubuntu 18.04, gdb-multiarch 8.1-0ubuntu3, gcc-arm-linux-gnueabihf 4:7.3.0-3ubuntu2, qemu-user 1:2.11+dfsg-1ubuntu7.3.
Edit: working pristine crosstool-ng setup
As a sanity check, I tried to get a clean toolchain with crosstool-ng, and it worked:
git clone https://github.com/crosstool-ng/crosstool-ng
cd crosstool-ng
git checkout d5900debd397b8909d9cafeb9a1093fb7a5dc6e6
export CT_PREFIX="$(pwd)/.build/ct_prefix"
./bootstrap
./configure --enable-local
./ct-ng arm-cortex_a15-linux-gnueabihf
# Has to be older than host kernel, which is 4.15.
printf "
CT_LINUX_V_4_14=y
CT_LINUX_VERSION=\"4.14.0\"
" >> .config
./ct-ng oldconfig
env -u LD_LIBRARY_PATH time ./ct-ng build -j`nproc`
cd ..
crosstool-ng/.build/ct_prefix/arm-cortex_a15-linux-gnueabihf/bin/arm-cortex_a15-linux-gnueabihf-gcc -ggdb3 -o hello_world hello_world.c -ggdb3 -static -o hello_world hello_world.c
qemu-arm -L crosstool-ng/.build/ct_prefix/arm-cortex_a15-linux-gnueabihf/arm-cortex_a15-linux-gnueabihf/sysroot -g 1234 ./hello_world
And on another shell:
./.build/ct_prefix/arm-cortex_a15-linux-gnueabihf/bin/arm-cortex_a15-linux-gnueabihf-gdb \
-q --nh \
-ex 'set architecture arm' \
-ex 'set sysroot crosstool-ng/.build/ct_prefix/arm-cortex_a15-linux-gnueabihf/arm-cortex_a15-linux-gnueabihf/sysroot' \
-ex 'file hello_world' \
-ex 'target remote localhost:1234' \
-ex 'break main' \
-ex continue \
;
And it also works with the distro provided gdb-multiarch.

Answer 1

The failure is due to -pie -fpie, and there is a bug report for it at: https://bugs.launchpad.net/qemu/+bug/1528239
Explicitly setting -no-pie -fno-pie makes it work on both crosstool-ng and Ubuntu host.
The difference is that the Ubuntu one has GCC built to use -fpie by default, while my crosstool-ng build did not.
This can be checked with:
gcc -v
which on the host GCC contains:
--enable-default-pie
but not in the crosstool-ng build.
How I found this out: the other day I was playing around with -fpie and I noticed that the .text addresses were really small in that case: What is the -fPIE option for position-independent executables in gcc and ld?
Then I saw that the break address was very small with the packaged toolchain, and made the link.