How to debug cross-compiled QEMU program with GDB? - gdb

Question

I'm having trouble debugging a simple program running in QEMU with GDB. GDB seems unable to find where I am in the program (in that it always displays ?? as my current location), and it never hits any breakpoint I set.
In one terminal, I run QEMU:
$ cat add.c
int main() {
int x = 9;
int v = 1;
while (1) {
int q = x + v;
}
return 0;
}
$ riscv64-unknown-elf-gcc add.c -g
$ qemu-system-riscv64 -gdb tcp::1234 -drive file=a.out,format=raw
And in another terminal, I run GDB:
$ riscv64-unknown-elf-gdb a.out
GNU gdb (GDB) 8.2.90.20190228-git
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-apple-darwin17.7.0 --target=riscv64-unknown-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from a.out...
(gdb) target remote :1234
Remote debugging using :1234
0x0000000000000000 in ?? ()
(gdb) list
1 int main() {
2 int x = 9;
3 int v = 1;
4 while (1) {
5 int q = x + v;
6 }
7 return 0;
8 }
(gdb) b main
Breakpoint 1 at 0x1018e: file add.c, line 2.
(gdb) b 5
Breakpoint 2 at 0x1019a: file add.c, line 5.
(gdb) b _start
Breakpoint 3 at 0x10114
(gdb) b 4
Breakpoint 4 at 0x101a8: file add.c, line 4.
(gdb) c
Continuing.
I never hit a breakpoint, even though the program should be looping infinitely. It seems odd that it's displaying 0x0000000000000000 in ?? ()...but maybe that's okay?
What am I doing wrong here? How can I step through this program?

Answer 1

I think you are missing a linker script and some startup code - disclaimer: I am a newcomer to riscv.
You will find a lot of information on those two topics on the Internet, but you basically need to specify where your program will be located in RAM, to establish a stack and initialize the frame pointer:
This is required if you want to be able to call functions and declare automatic C variables like a, b, c in your program.
I used the Windows toolchain from Kendryte for the purpose of this example (the Linux version is available here), and a Windows version of qemu retrieved here.
1) Linker script: the example uses a slightly modified example of the default linker script used by riscv64-unknown-elf-ld:
riscv64-unknown-elf-ld --verbose > riscv64-virt.ld
Edit riscv64-virt.ld, and keep only the lines delimited by:
==================================================
Add a description for the memory layout of the qemu-system-riscv64 virt machine:
OUTPUT_ARCH(riscv)
MEMORY
{
/* qemu-system-risc64 virt machine */
RAM (rwx) : ORIGIN = 0x80000000, LENGTH = 128M
}
ENTRY(_start)
Use ORIGIN(RAM) and LENGTH(RAM) instead of hard-coded values, and provide a __stack_top symbol:
PROVIDE (__executable_start = SEGMENT_START("text-segment", ORIGIN(RAM))); . = SEGMENT_START("text-segment", ORIGIN(RAM)) + SIZEOF_HEADERS;
PROVIDE(__stack_top = ORIGIN(RAM) + LENGTH(RAM));
By the way, there are multiple ways of learning the memory layout of a qemu-system target machine, but I usually look at its Device Tree file:
qemu-system-riscv64 -machine virt -machine dumpdtb=riscv64-virt.dtb
dtc -I dtb -O dts -o riscv-virt.dts riscv-virt.dtb
The section describing the memory tells us it starts at 0x80000000:
memory#80000000 {
device_type = "memory";
reg = <0x0 0x80000000 0x0 0x8000000>;
};
riscv64-virt.ld:
/* Script for -z combreloc: combine and sort reloc sections */
/* Copyright (C) 2014-2018 Free Software Foundation, Inc.
Copying and distribution of this script, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved. */
OUTPUT_FORMAT("elf64-littleriscv", "elf64-littleriscv",
"elf64-littleriscv")
OUTPUT_ARCH(riscv)
MEMORY
{
/* qemu-system-risc64 virt machine */
RAM (rwx) : ORIGIN = 0x80000000, LENGTH = 128M
}
ENTRY(_start)
SECTIONS
{
/* Read-only sections, merged into text segment: */
PROVIDE (__executable_start = SEGMENT_START("text-segment", ORIGIN(RAM))); . = SEGMENT_START("text-segment", ORIGIN(RAM)) + SIZEOF_HEADERS;
PROVIDE(__stack_top = ORIGIN(RAM) + LENGTH(RAM));
.interp : { *(.interp) }
.note.gnu.build-id : { *(.note.gnu.build-id) }
.hash : { *(.hash) }
.gnu.hash : { *(.gnu.hash) }
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
.gnu.version : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.rela.dyn :
{
*(.rela.init)
*(.rela.text .rela.text.* .rela.gnu.linkonce.t.*)
*(.rela.fini)
*(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*)
*(.rela.data .rela.data.* .rela.gnu.linkonce.d.*)
*(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*)
*(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*)
*(.rela.ctors)
*(.rela.dtors)
*(.rela.got)
*(.rela.sdata .rela.sdata.* .rela.gnu.linkonce.s.*)
*(.rela.sbss .rela.sbss.* .rela.gnu.linkonce.sb.*)
*(.rela.sdata2 .rela.sdata2.* .rela.gnu.linkonce.s2.*)
*(.rela.sbss2 .rela.sbss2.* .rela.gnu.linkonce.sb2.*)
*(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*)
PROVIDE_HIDDEN (__rela_iplt_start = .);
*(.rela.iplt)
PROVIDE_HIDDEN (__rela_iplt_end = .);
}
.rela.plt :
{
*(.rela.plt)
}
.init :
{
KEEP (*(SORT_NONE(.init)))
}
.plt : { *(.plt) }
.iplt : { *(.iplt) }
.text :
{
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
/* .gnu.warning sections are handled specially by elf32.em. */
*(.gnu.warning)
}
.fini :
{
KEEP (*(SORT_NONE(.fini)))
}
PROVIDE (__etext = .);
PROVIDE (_etext = .);
PROVIDE (etext = .);
.rodata : { *(.rodata .rodata.* .gnu.linkonce.r.*) }
.rodata1 : { *(.rodata1) }
.sdata2 :
{
*(.sdata2 .sdata2.* .gnu.linkonce.s2.*)
}
.sbss2 : { *(.sbss2 .sbss2.* .gnu.linkonce.sb2.*) }
.eh_frame_hdr : { *(.eh_frame_hdr) *(.eh_frame_entry .eh_frame_entry.*) }
.eh_frame : ONLY_IF_RO { KEEP (*(.eh_frame)) *(.eh_frame.*) }
.gcc_except_table : ONLY_IF_RO { *(.gcc_except_table
.gcc_except_table.*) }
.gnu_extab : ONLY_IF_RO { *(.gnu_extab*) }
/* These sections are generated by the Sun/Oracle C++ compiler. */
.exception_ranges : ONLY_IF_RO { *(.exception_ranges
.exception_ranges*) }
/* Adjust the address for the data segment. We want to adjust up to
the same address within the page on the next page up. */
. = DATA_SEGMENT_ALIGN (CONSTANT (MAXPAGESIZE), CONSTANT (COMMONPAGESIZE));
/* Exception handling */
.eh_frame : ONLY_IF_RW { KEEP (*(.eh_frame)) *(.eh_frame.*) }
.gnu_extab : ONLY_IF_RW { *(.gnu_extab) }
.gcc_except_table : ONLY_IF_RW { *(.gcc_except_table .gcc_except_table.*) }
.exception_ranges : ONLY_IF_RW { *(.exception_ranges .exception_ranges*) }
/* Thread Local Storage sections */
.tdata :
{
PROVIDE_HIDDEN (__tdata_start = .);
*(.tdata .tdata.* .gnu.linkonce.td.*)
}
.tbss : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
.preinit_array :
{
PROVIDE_HIDDEN (__preinit_array_start = .);
KEEP (*(.preinit_array))
PROVIDE_HIDDEN (__preinit_array_end = .);
}
.init_array :
{
PROVIDE_HIDDEN (__init_array_start = .);
KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP (*(.init_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .ctors))
PROVIDE_HIDDEN (__init_array_end = .);
}
.fini_array :
{
PROVIDE_HIDDEN (__fini_array_start = .);
KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP (*(.fini_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .dtors))
PROVIDE_HIDDEN (__fini_array_end = .);
}
.ctors :
{
/* gcc uses crtbegin.o to find the start of
the constructors, so we make sure it is
first. Because this is a wildcard, it
doesn't matter if the user does not
actually link against crtbegin.o; the
linker won't look for a file to match a
wildcard. The wildcard also means that it
doesn't matter which directory crtbegin.o
is in. */
KEEP (*crtbegin.o(.ctors))
KEEP (*crtbegin?.o(.ctors))
/* We don't want to include the .ctor section from
the crtend.o file until after the sorted ctors.
The .ctor section from the crtend file contains the
end of ctors marker and it must be last */
KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .ctors))
KEEP (*(SORT(.ctors.*)))
KEEP (*(.ctors))
}
.dtors :
{
KEEP (*crtbegin.o(.dtors))
KEEP (*crtbegin?.o(.dtors))
KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .dtors))
KEEP (*(SORT(.dtors.*)))
KEEP (*(.dtors))
}
.jcr : { KEEP (*(.jcr)) }
.data.rel.ro : { *(.data.rel.ro.local* .gnu.linkonce.d.rel.ro.local.*) *(.data.rel.ro .data.rel.ro.* .gnu.linkonce.d.rel.ro.*) }
.dynamic : { *(.dynamic) }
. = DATA_SEGMENT_RELRO_END (0, .);
.data :
{
*(.data .data.* .gnu.linkonce.d.*)
SORT(CONSTRUCTORS)
}
.data1 : { *(.data1) }
.got : { *(.got.plt) *(.igot.plt) *(.got) *(.igot) }
/* We want the small data sections together, so single-instruction offsets
can access them all, and initialized data all before uninitialized, so
we can shorten the on-disk segment size. */
.sdata :
{
__global_pointer$ = . + 0x800;
*(.srodata.cst16) *(.srodata.cst8) *(.srodata.cst4) *(.srodata.cst2) *(.srodata .srodata.*)
*(.sdata .sdata.* .gnu.linkonce.s.*)
}
_edata = .; PROVIDE (edata = .);
. = .;
__bss_start = .;
.sbss :
{
*(.dynsbss)
*(.sbss .sbss.* .gnu.linkonce.sb.*)
*(.scommon)
}
.bss :
{
*(.dynbss)
*(.bss .bss.* .gnu.linkonce.b.*)
*(COMMON)
/* Align here to ensure that the .bss section occupies space up to
_end. Align after .bss to ensure correct alignment even if the
.bss section disappears because there are no input sections.
FIXME: Why do we need it? When there is no .bss section, we don't
pad the .data section. */
. = ALIGN(. != 0 ? 64 / 8 : 1);
}
. = ALIGN(64 / 8);
. = SEGMENT_START("ldata-segment", .);
. = ALIGN(64 / 8);
_end = .; PROVIDE (end = .);
. = DATA_SEGMENT_END (.);
/* Stabs debugging sections. */
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.comment 0 : { *(.comment) }
/* DWARF debug sections.
Symbols in the DWARF debugging sections are relative to the beginning
of the section so we begin them at 0. */
/* DWARF 1 */
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
/* GNU DWARF 1 extensions */
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
/* DWARF 1.1 and DWARF 2 */
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
/* DWARF 2 */
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end ) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
/* SGI/MIPS DWARF 2 extensions */
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
/* DWARF 3 */
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
/* DWARF Extension. */
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.gnu.attributes 0 : { KEEP (*(.gnu.attributes)) }
/DISCARD/ : { *(.note.GNU-stack) *(.gnu_debuglink) *(.gnu.lto_*) }
}
2) startup.s: (credits: here and here).
.section .init, "ax"
.global _start
_start:
.cfi_startproc
.cfi_undefined ra
.option push
.option norelax
la gp, __global_pointer$
.option pop
la sp, __stack_top
add s0, sp, zero
jal zero, main
.cfi_endproc
.end
add.c: (your code)
int main() {
int a = 4;
int b = 12;
while (1) {
int c = a + b;
}
return 0;
}
3) compiling/linking, and creating a listing:
riscv64-unknown-elf-gcc -g -ffreestanding -O0 -Wl,--gc-sections -nostartfiles -nostdlib -nodefaultlibs -Wl,-T,riscv64-virt.ld -o add.elf startup.s add.c
riscv64-unknown-elf-objdump -D add.elf > add.objdump
4) starting qemu in a console:
qemu-system-riscv64 -machine virt -m 128M -gdb tcp::1234,ipv4 -kernel add.elf
I am not sure that the qemu options you were using: -drive file=a.out,format=raw
are correct, and I think they are not, but I did not spend time checking, and used the options I am usually using: -kernel add.elf
4) starting gdb in another console (I am using here a GDB I compiled with TUI support for mingw64 for my own convenience).
riscv64-elf-gdb --tui add.elf
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
main () at add.c:5
(gdb) p a
$1 = 4
(gdb) p b
$2 = 12
(gdb) p c
$3 = 16
(gdb)
This may have been a little bit long, but I hope this will help.
Please note that the startup code is good enough for your code, but some important initializations are missing, such as copying the data section from flash to RAM (not relevant here), and clearing the .bss section.

Answer 2

Using riscv-gnu-toolchain built with glibc is a much simpler method to debug riscv programs unless you are debugging some system-level program where you must use riscv64-unknown-elf-gcc instead of riscv64-unknown-linux-gnu-gcc. For a simple program like your add.c, using user-space qemu-riscv and glibc riscv-gnu-toolchain can save you a lot of trouble. (One can install these tools following the commands listed at the bottom)
By the time I am writing this answer, there are two different versions of the riscv toolchain: one built with newlib which provides riscv64-unknown-elf-* and another with glibc which provides riscv64-unknown-linux-gnu-*. There are also two versions of qemu: qemu-system-riscv64 for debugging kernels or bare-metal programs and qemu-riscv64 for debugging user-space programs compiled with libc.
For simple programs like add.c, one may debug it with the second type of the toolchain:
Compile: riscv64-unknown-linux-gnu-gcc add.c -o add -g
Run: qemu-riscv64 -L /opt/riscv/sysroot/ -g 1234 add -S
Then launch GDB: riscv64-unknown-linux-gnu-gdb add
Inside GDB:
target remote:1234
b main
c
And the program should break at the main entrance.
(Another option is to statically link the program: riscv64-unknown-linux-gnu-gcc add.c -o add -g -static and then qemu-riscv64 -g 1234 add -S should work as well)
I did not find many documents mentioning user-space riscv qemu. All I found were articles talking about how to use qemu to debug OS kernels with RISC-V ISA. For the convenience of other newcomers to riscv like me, I will show in the following how to build the mentioned tools.
qemu (https://www.qemu.org/download/#source)
wget https://download.qemu.org/qemu-5.0.0.tar.xz
tar xvJf qemu-5.0.0.tar.xz
cd qemu-5.0.0 # higher versions might have problems
./configure --target-list=riscv64-linux-user,riscv64-softmmu
make -j$(nproc)
sudo make install
, where riscv64-softmmu gives you qemu-system-riscv64 and riscv64-linux-user gives you qemu-riscv64.
riscv-gnu-toolchain (https://github.com/riscv/riscv-gnu-toolchain.git)
git clone https://github.com/riscv/riscv-gnu-toolchain.git
sudo apt-get install autoconf automake autotools-dev curl python3 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev libncurses5-dev
./configure --prefix=/opt/riscv --enable-multilib
sudo make linux # this provides you the glibc set of tools (the ones we need here)
sudo make # this provides you the newlib set of tools