GNU LD section attributes and flags for 64-bit kernel ELF - c++

I am trying to link a 64-bit kernel ELF using GNU LD. I have a executable section named lowerhalf and then the other usual sections. The linker script I use is this
ENTRY(kernelCompatibilityModeStart);
SECTIONS {
. = 0x80000000; /* lowerhalf origin */
.lowerhalf : ALIGN(0x1000) {
*(.lowerhalf);
}
. = 0xffffffff80000000; /* higherhalf origin */
.GDT64 : ALIGN(0x1000) {
*(.GDT64);
}
.IDT64 : ALIGN(0x1000) {
*(.IDT64);
}
.TSS64 : ALIGN(0x1000) {
*(.TSS64);
}
.KERNELSTACK : ALIGN(0x1000) {
*(.KERNELSTACK);
}
.ISTs : ALIGN(0x1000) {
*(.ISTs);
}
.text : ALIGN(0x1000) {
*(.text);
}
.data : ALIGN(0x1000) {
*(.data);
}
.rodata : ALIGN(0x1000) {
*(.rodata*);
}
.bss : ALIGN(0x1000) {
*(COMMON);
*(.bss);
}
}
When I look at the generated segments using readelf I see something like
Elf file type is EXEC (Executable file)
Entry point 0x80000000
There are 3 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000001000 0x0000000080000000 0x0000000080000000
0x0000000000000030 0x0000000000000030 R 0x1000
LOAD 0x0000000000002000 0xffffffff80000000 0xffffffff80000000
0x0000000000026718 0x0000000000026718 R E 0x1000
LOAD 0x0000000000029000 0xffffffff80027000 0xffffffff80027000
0x0000000000004954 0x00000000000053d8 RW 0x1000
Section to Segment mapping:
Segment Sections...
00 .lowerhalf
01 .GDT64 .IDT64 .TSS64 .KERNELSTACK .ISTs .text
02 .data .ctors .rodata .eh_frame .bss
Some problems with the output ELF:
the .text section gets bundled up with other sections like .GDT64 and .KERNELSTACK
.lowerhalf should have RE attributes instead of just R
all .data, .ctors, and .rodata end up with the RW flags.
I tried using the MEMORY directive but that results in a load of linker errors or warnings and doesn't give the desired output.
I'd like for the sections to have appropriate RWE attributes so I can create my paging entries accordingly.

Related

Why does the dynamic linker *subtract* the virtual address to find out location of loaded shared library executable in memory?

According to the ld source code here in dl_main. Without the context of what phdr is passed in to dl_main, I'm a bit confused on why main_map's load address is deduced by subtracting the virtual address.
The code that I've traced through:
1124 static void
1125 dl_main (const ElfW(Phdr) *phdr,
1126 ElfW(Word) phnum,
1127 ElfW(Addr) *user_entry,
1128 ElfW(auxv_t) *auxv)
1129 {
1130 const ElfW(Phdr) *ph;
1131 enum mode mode;
1132 struct link_map *main_map;
1133 size_t file_size;
1134 char *file;
1135 bool has_interp = false;
1136 unsigned int i;
1137 bool prelinked = false;
1138 bool rtld_is_main = false;
1139 void *tcbp = NULL;
... // Before this else, it thinks you're calling `ld.so.<version>` directly. This is not usually the case.
1366 else
1367 {
1368 /* Create a link_map for the executable itself.
1369 This will be what dlopen on "" returns. */
1370 main_map = _dl_new_object ((char *) "", "", lt_executable, NULL,
1371 __RTLD_OPENEXEC, LM_ID_BASE);
1372 assert (main_map != NULL);
1373 main_map->l_phdr = phdr;
1374 main_map->l_phnum = phnum;
1375 main_map->l_entry = *user_entry;
1376
1377 /* Even though the link map is not yet fully initialized we can add
1378 it to the map list since there are no possible users running yet. */
1379 _dl_add_to_namespace_list (main_map, LM_ID_BASE);
1380 assert (main_map == GL(dl_ns)[LM_ID_BASE]._ns_loaded);
1399 }
... // Loops through program headers loaded in sequence from the ELF header.
1409 for (ph = phdr; ph < &phdr[phnum]; ++ph)
1410 switch (ph->p_type)
1411 {
1412 case PT_PHDR:
1413 /* Find out the load address. */
1414 main_map->l_addr = (ElfW(Addr)) phdr - ph->p_vaddr;
1415 break;
So why is the load address subtracted here?
In particular, on line 1414, we see main_map->l_addr = (ElfW(Addr)) phdr - ph->p_vaddr;. From the file link.h which defines the type link_map for main_map, I see that link_map is a struct that describes a loaded shared object. The l_addr field is to describe the difference between where the .so is loaded into memory vs. where it says it is loaded in the VirtAddr field when you run readelf:
❯ readelf -l main
Elf file type is EXEC (Executable file)
Entry point 0x401020
There are 11 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040
0x0000000000000268 0x0000000000000268 R 0x8
INTERP 0x00000000000002a8 0x00000000004002a8 0x00000000004002a8
0x000000000000001c 0x000000000000001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x00000000000004c0 0x00000000000004c0 R 0x1000
...
This means that l_addr is NOT the load address. It's actually the offset for which you need to add from the current memory to access the shared object contents.

How to debug cross-compiled QEMU program with GDB?

I'm having trouble debugging a simple program running in QEMU with GDB. GDB seems unable to find where I am in the program (in that it always displays ?? as my current location), and it never hits any breakpoint I set.
In one terminal, I run QEMU:
$ cat add.c
int main() {
int x = 9;
int v = 1;
while (1) {
int q = x + v;
}
return 0;
}
$ riscv64-unknown-elf-gcc add.c -g
$ qemu-system-riscv64 -gdb tcp::1234 -drive file=a.out,format=raw
And in another terminal, I run GDB:
$ riscv64-unknown-elf-gdb a.out
GNU gdb (GDB) 8.2.90.20190228-git
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-apple-darwin17.7.0 --target=riscv64-unknown-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from a.out...
(gdb) target remote :1234
Remote debugging using :1234
0x0000000000000000 in ?? ()
(gdb) list
1 int main() {
2 int x = 9;
3 int v = 1;
4 while (1) {
5 int q = x + v;
6 }
7 return 0;
8 }
(gdb) b main
Breakpoint 1 at 0x1018e: file add.c, line 2.
(gdb) b 5
Breakpoint 2 at 0x1019a: file add.c, line 5.
(gdb) b _start
Breakpoint 3 at 0x10114
(gdb) b 4
Breakpoint 4 at 0x101a8: file add.c, line 4.
(gdb) c
Continuing.
I never hit a breakpoint, even though the program should be looping infinitely. It seems odd that it's displaying 0x0000000000000000 in ?? ()...but maybe that's okay?
What am I doing wrong here? How can I step through this program?
I think you are missing a linker script and some startup code - disclaimer: I am a newcomer to riscv.
You will find a lot of information on those two topics on the Internet, but you basically need to specify where your program will be located in RAM, to establish a stack and initialize the frame pointer:
This is required if you want to be able to call functions and declare automatic C variables like a, b, c in your program.
I used the Windows toolchain from Kendryte for the purpose of this example (the Linux version is available here), and a Windows version of qemu retrieved here.
1) Linker script: the example uses a slightly modified example of the default linker script used by riscv64-unknown-elf-ld:
riscv64-unknown-elf-ld --verbose > riscv64-virt.ld
Edit riscv64-virt.ld, and keep only the lines delimited by:
==================================================
Add a description for the memory layout of the qemu-system-riscv64 virt machine:
OUTPUT_ARCH(riscv)
MEMORY
{
/* qemu-system-risc64 virt machine */
RAM (rwx) : ORIGIN = 0x80000000, LENGTH = 128M
}
ENTRY(_start)
Use ORIGIN(RAM) and LENGTH(RAM) instead of hard-coded values, and provide a __stack_top symbol:
PROVIDE (__executable_start = SEGMENT_START("text-segment", ORIGIN(RAM))); . = SEGMENT_START("text-segment", ORIGIN(RAM)) + SIZEOF_HEADERS;
PROVIDE(__stack_top = ORIGIN(RAM) + LENGTH(RAM));
By the way, there are multiple ways of learning the memory layout of a qemu-system target machine, but I usually look at its Device Tree file:
qemu-system-riscv64 -machine virt -machine dumpdtb=riscv64-virt.dtb
dtc -I dtb -O dts -o riscv-virt.dts riscv-virt.dtb
The section describing the memory tells us it starts at 0x80000000:
memory#80000000 {
device_type = "memory";
reg = <0x0 0x80000000 0x0 0x8000000>;
};
riscv64-virt.ld:
/* Script for -z combreloc: combine and sort reloc sections */
/* Copyright (C) 2014-2018 Free Software Foundation, Inc.
Copying and distribution of this script, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved. */
OUTPUT_FORMAT("elf64-littleriscv", "elf64-littleriscv",
"elf64-littleriscv")
OUTPUT_ARCH(riscv)
MEMORY
{
/* qemu-system-risc64 virt machine */
RAM (rwx) : ORIGIN = 0x80000000, LENGTH = 128M
}
ENTRY(_start)
SECTIONS
{
/* Read-only sections, merged into text segment: */
PROVIDE (__executable_start = SEGMENT_START("text-segment", ORIGIN(RAM))); . = SEGMENT_START("text-segment", ORIGIN(RAM)) + SIZEOF_HEADERS;
PROVIDE(__stack_top = ORIGIN(RAM) + LENGTH(RAM));
.interp : { *(.interp) }
.note.gnu.build-id : { *(.note.gnu.build-id) }
.hash : { *(.hash) }
.gnu.hash : { *(.gnu.hash) }
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
.gnu.version : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.rela.dyn :
{
*(.rela.init)
*(.rela.text .rela.text.* .rela.gnu.linkonce.t.*)
*(.rela.fini)
*(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*)
*(.rela.data .rela.data.* .rela.gnu.linkonce.d.*)
*(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*)
*(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*)
*(.rela.ctors)
*(.rela.dtors)
*(.rela.got)
*(.rela.sdata .rela.sdata.* .rela.gnu.linkonce.s.*)
*(.rela.sbss .rela.sbss.* .rela.gnu.linkonce.sb.*)
*(.rela.sdata2 .rela.sdata2.* .rela.gnu.linkonce.s2.*)
*(.rela.sbss2 .rela.sbss2.* .rela.gnu.linkonce.sb2.*)
*(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*)
PROVIDE_HIDDEN (__rela_iplt_start = .);
*(.rela.iplt)
PROVIDE_HIDDEN (__rela_iplt_end = .);
}
.rela.plt :
{
*(.rela.plt)
}
.init :
{
KEEP (*(SORT_NONE(.init)))
}
.plt : { *(.plt) }
.iplt : { *(.iplt) }
.text :
{
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
/* .gnu.warning sections are handled specially by elf32.em. */
*(.gnu.warning)
}
.fini :
{
KEEP (*(SORT_NONE(.fini)))
}
PROVIDE (__etext = .);
PROVIDE (_etext = .);
PROVIDE (etext = .);
.rodata : { *(.rodata .rodata.* .gnu.linkonce.r.*) }
.rodata1 : { *(.rodata1) }
.sdata2 :
{
*(.sdata2 .sdata2.* .gnu.linkonce.s2.*)
}
.sbss2 : { *(.sbss2 .sbss2.* .gnu.linkonce.sb2.*) }
.eh_frame_hdr : { *(.eh_frame_hdr) *(.eh_frame_entry .eh_frame_entry.*) }
.eh_frame : ONLY_IF_RO { KEEP (*(.eh_frame)) *(.eh_frame.*) }
.gcc_except_table : ONLY_IF_RO { *(.gcc_except_table
.gcc_except_table.*) }
.gnu_extab : ONLY_IF_RO { *(.gnu_extab*) }
/* These sections are generated by the Sun/Oracle C++ compiler. */
.exception_ranges : ONLY_IF_RO { *(.exception_ranges
.exception_ranges*) }
/* Adjust the address for the data segment. We want to adjust up to
the same address within the page on the next page up. */
. = DATA_SEGMENT_ALIGN (CONSTANT (MAXPAGESIZE), CONSTANT (COMMONPAGESIZE));
/* Exception handling */
.eh_frame : ONLY_IF_RW { KEEP (*(.eh_frame)) *(.eh_frame.*) }
.gnu_extab : ONLY_IF_RW { *(.gnu_extab) }
.gcc_except_table : ONLY_IF_RW { *(.gcc_except_table .gcc_except_table.*) }
.exception_ranges : ONLY_IF_RW { *(.exception_ranges .exception_ranges*) }
/* Thread Local Storage sections */
.tdata :
{
PROVIDE_HIDDEN (__tdata_start = .);
*(.tdata .tdata.* .gnu.linkonce.td.*)
}
.tbss : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
.preinit_array :
{
PROVIDE_HIDDEN (__preinit_array_start = .);
KEEP (*(.preinit_array))
PROVIDE_HIDDEN (__preinit_array_end = .);
}
.init_array :
{
PROVIDE_HIDDEN (__init_array_start = .);
KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP (*(.init_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .ctors))
PROVIDE_HIDDEN (__init_array_end = .);
}
.fini_array :
{
PROVIDE_HIDDEN (__fini_array_start = .);
KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP (*(.fini_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .dtors))
PROVIDE_HIDDEN (__fini_array_end = .);
}
.ctors :
{
/* gcc uses crtbegin.o to find the start of
the constructors, so we make sure it is
first. Because this is a wildcard, it
doesn't matter if the user does not
actually link against crtbegin.o; the
linker won't look for a file to match a
wildcard. The wildcard also means that it
doesn't matter which directory crtbegin.o
is in. */
KEEP (*crtbegin.o(.ctors))
KEEP (*crtbegin?.o(.ctors))
/* We don't want to include the .ctor section from
the crtend.o file until after the sorted ctors.
The .ctor section from the crtend file contains the
end of ctors marker and it must be last */
KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .ctors))
KEEP (*(SORT(.ctors.*)))
KEEP (*(.ctors))
}
.dtors :
{
KEEP (*crtbegin.o(.dtors))
KEEP (*crtbegin?.o(.dtors))
KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .dtors))
KEEP (*(SORT(.dtors.*)))
KEEP (*(.dtors))
}
.jcr : { KEEP (*(.jcr)) }
.data.rel.ro : { *(.data.rel.ro.local* .gnu.linkonce.d.rel.ro.local.*) *(.data.rel.ro .data.rel.ro.* .gnu.linkonce.d.rel.ro.*) }
.dynamic : { *(.dynamic) }
. = DATA_SEGMENT_RELRO_END (0, .);
.data :
{
*(.data .data.* .gnu.linkonce.d.*)
SORT(CONSTRUCTORS)
}
.data1 : { *(.data1) }
.got : { *(.got.plt) *(.igot.plt) *(.got) *(.igot) }
/* We want the small data sections together, so single-instruction offsets
can access them all, and initialized data all before uninitialized, so
we can shorten the on-disk segment size. */
.sdata :
{
__global_pointer$ = . + 0x800;
*(.srodata.cst16) *(.srodata.cst8) *(.srodata.cst4) *(.srodata.cst2) *(.srodata .srodata.*)
*(.sdata .sdata.* .gnu.linkonce.s.*)
}
_edata = .; PROVIDE (edata = .);
. = .;
__bss_start = .;
.sbss :
{
*(.dynsbss)
*(.sbss .sbss.* .gnu.linkonce.sb.*)
*(.scommon)
}
.bss :
{
*(.dynbss)
*(.bss .bss.* .gnu.linkonce.b.*)
*(COMMON)
/* Align here to ensure that the .bss section occupies space up to
_end. Align after .bss to ensure correct alignment even if the
.bss section disappears because there are no input sections.
FIXME: Why do we need it? When there is no .bss section, we don't
pad the .data section. */
. = ALIGN(. != 0 ? 64 / 8 : 1);
}
. = ALIGN(64 / 8);
. = SEGMENT_START("ldata-segment", .);
. = ALIGN(64 / 8);
_end = .; PROVIDE (end = .);
. = DATA_SEGMENT_END (.);
/* Stabs debugging sections. */
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.comment 0 : { *(.comment) }
/* DWARF debug sections.
Symbols in the DWARF debugging sections are relative to the beginning
of the section so we begin them at 0. */
/* DWARF 1 */
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
/* GNU DWARF 1 extensions */
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
/* DWARF 1.1 and DWARF 2 */
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
/* DWARF 2 */
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end ) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
/* SGI/MIPS DWARF 2 extensions */
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
/* DWARF 3 */
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
/* DWARF Extension. */
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.gnu.attributes 0 : { KEEP (*(.gnu.attributes)) }
/DISCARD/ : { *(.note.GNU-stack) *(.gnu_debuglink) *(.gnu.lto_*) }
}
2) startup.s: (credits: here and here).
.section .init, "ax"
.global _start
_start:
.cfi_startproc
.cfi_undefined ra
.option push
.option norelax
la gp, __global_pointer$
.option pop
la sp, __stack_top
add s0, sp, zero
jal zero, main
.cfi_endproc
.end
add.c: (your code)
int main() {
int a = 4;
int b = 12;
while (1) {
int c = a + b;
}
return 0;
}
3) compiling/linking, and creating a listing:
riscv64-unknown-elf-gcc -g -ffreestanding -O0 -Wl,--gc-sections -nostartfiles -nostdlib -nodefaultlibs -Wl,-T,riscv64-virt.ld -o add.elf startup.s add.c
riscv64-unknown-elf-objdump -D add.elf > add.objdump
4) starting qemu in a console:
qemu-system-riscv64 -machine virt -m 128M -gdb tcp::1234,ipv4 -kernel add.elf
I am not sure that the qemu options you were using: -drive file=a.out,format=raw
are correct, and I think they are not, but I did not spend time checking, and used the options I am usually using: -kernel add.elf
4) starting gdb in another console (I am using here a GDB I compiled with TUI support for mingw64 for my own convenience).
riscv64-elf-gdb --tui add.elf
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
main () at add.c:5
(gdb) p a
$1 = 4
(gdb) p b
$2 = 12
(gdb) p c
$3 = 16
(gdb)
This may have been a little bit long, but I hope this will help.
Please note that the startup code is good enough for your code, but some important initializations are missing, such as copying the data section from flash to RAM (not relevant here), and clearing the .bss section.
Using riscv-gnu-toolchain built with glibc is a much simpler method to debug riscv programs unless you are debugging some system-level program where you must use riscv64-unknown-elf-gcc instead of riscv64-unknown-linux-gnu-gcc. For a simple program like your add.c, using user-space qemu-riscv and glibc riscv-gnu-toolchain can save you a lot of trouble. (One can install these tools following the commands listed at the bottom)
By the time I am writing this answer, there are two different versions of the riscv toolchain: one built with newlib which provides riscv64-unknown-elf-* and another with glibc which provides riscv64-unknown-linux-gnu-*. There are also two versions of qemu: qemu-system-riscv64 for debugging kernels or bare-metal programs and qemu-riscv64 for debugging user-space programs compiled with libc.
For simple programs like add.c, one may debug it with the second type of the toolchain:
Compile: riscv64-unknown-linux-gnu-gcc add.c -o add -g
Run: qemu-riscv64 -L /opt/riscv/sysroot/ -g 1234 add -S
Then launch GDB: riscv64-unknown-linux-gnu-gdb add
Inside GDB:
target remote:1234
b main
c
And the program should break at the main entrance.
(Another option is to statically link the program: riscv64-unknown-linux-gnu-gcc add.c -o add -g -static and then qemu-riscv64 -g 1234 add -S should work as well)
I did not find many documents mentioning user-space riscv qemu. All I found were articles talking about how to use qemu to debug OS kernels with RISC-V ISA. For the convenience of other newcomers to riscv like me, I will show in the following how to build the mentioned tools.
qemu (https://www.qemu.org/download/#source)
wget https://download.qemu.org/qemu-5.0.0.tar.xz
tar xvJf qemu-5.0.0.tar.xz
cd qemu-5.0.0 # higher versions might have problems
./configure --target-list=riscv64-linux-user,riscv64-softmmu
make -j$(nproc)
sudo make install
, where riscv64-softmmu gives you qemu-system-riscv64 and riscv64-linux-user gives you qemu-riscv64.
riscv-gnu-toolchain (https://github.com/riscv/riscv-gnu-toolchain.git)
git clone https://github.com/riscv/riscv-gnu-toolchain.git
sudo apt-get install autoconf automake autotools-dev curl python3 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev libncurses5-dev
./configure --prefix=/opt/riscv --enable-multilib
sudo make linux # this provides you the glibc set of tools (the ones we need here)
sudo make # this provides you the newlib set of tools

Linking a 64 bit Kernel

Hi All I have been trying to link my assembly code to a C++ file so I can fall my function kMain from assembly and When I link it with this script :
ENTRY(_Start)
SECTIONS
{
. = 0x2000;
.text : AT(ADDR(.text) - 0x2000)
{
_code = .;
*(.text)
*(.rodata*)
. = ALIGN(4096);
}
.data : AT(ADDR(.data) - 0x2000)
{
_data = .;
*(.data)
. = ALIGN(4096);
}
.eh_frame : AT(ADDR(.eh_frame) - 0x2000)
{
_ehframe = .;
*(.eh_frame)
. = ALIGN(4096);
}
.bss : AT(ADDR(.bss) - 0x2000)
{
_bss = .;
*(.bss)
/*
* You usually need to include generated COMMON symbols
* under kernel BSS section or use gcc's -fno-common
*/
*(COMMON)
. = ALIGN(4096);
}
_end = .;
/DISCARD/ :
{
*(.comment)
}
}
I get a warning saying : x86_64-elf-ld: warning: cannot find entry symbol _Start; defaulting to 0000000000002000
But in my Assembly Code I have this at the start:
[BITS 16]
_Start:
Any Ideas as to why its not linking Correctly??
EDIT: It works now With this declared:
global _Start:
_Start:
But It won't load the program at the adress 0x2000
I use a batch program to compile/assemble, format and link my OS here it is:
nasm Stage1.asm -o Stage1.bin
nasm -f elf64 Stage2.asm -o Stage2.o
x86_64-elf-g++ -ffreestanding -mcmodel=large -mno-red-zone -mno-mmx -mno-sse -mno-sse2 -mno-sse3 -mno-3dnow -c -o kernel.o kernel.cpp
x86_64-elf-ld -T linkerscript.ld -o MyOS.bin Stage2.o kernel.o -nostdlib
copy Stage1.bin Root
copy MyOS.bin Root
mkisofs -b Stage1.bin -no-emul-boot -boot-info-table -o BootLoader.iso ./Root
If you wan't to see all of the source code it is here:
https://github.com/AnonymousUser1337/Anmu
You probably have to declare the _Start label as global using some kind of assembler directive (e.g. global).

Bad align value for a ELF section causes the program to be loaded wrong

I'm currently building a toy OS using a custom linker script to create the binary :
ENTRY(entry_point)
/* base virtual address of the kernel */
VIRT_BASE = 0xFFFFFFFF80000000;
SECTIONS
{
. = 0x100000;
/*
* Place multiboot header at 0x10000 as it is where Grub will be looking
* for it.
* Immediately followed by the boot code
*/
.boot :
{
*(.mbhdr)
_load_start = .;
*(.boot)
. = ALIGN(4096);
/* reserve space for paging data structures */
pml4 = .;
. += 0x1000;
pdpt = .;
. += 0x1000;
pagedir = .;
. += 0x1000;
. += 0x8000;
/* stack segment for loader */
stack = .;
}
/*
* Kernel code section is placed at his virtual address
*/
. += VIRT_BASE;
.text ALIGN(0x1000) : AT(ADDR(.text) - VIRT_BASE)
{
*(.text)
*(.gnu.linkonce.t*)
}
.data ALIGN(0x1000) : AT(ADDR(.data) - VIRT_BASE)
{
*(.data)
*(.gnu.linkonce.d*)
}
.rodata ALIGN(0x1000) : AT(ADDR(.rodata) - VIRT_BASE)
{
*(.rodata*)
*(.gnu.linkonce.r*)
}
_load_end = . - VIRT_BASE;
.bss ALIGN(0x1000) : AT(ADDR(.bss) - VIRT_BASE)
{
*(COMMON)
*(.bss)
*(.gnu.linkonce.b*)
}
_bss_end = . - VIRT_BASE;
/DISCARD/ :
{
*(.comment)
*(.eh_frame)
}
}
Since I use virtual memory, I use the AT() directive to make a distinction between the relocation address (the value of the symbol) and the actual physical load address, because virtual memory is not enabled when the binary is loaded. The address I give to AT correspond to the physical address mapped to the virtual relocation address. this works well.
But in some cases, I notice there is a strange shift after my code is loaded in memory. the .text section (which does not include the assembly boot code, only C++ code) is located 8 bytes higher than where it should be. An objdump shows the virtual relocation address is correct in the binary, as well as the load address. The only visible change is in readelf -l :
Elf file type is EXEC (Executable file)
Entry point 0x10003c
There are 3 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x0000e8 0x0000000000100000 0x0000000000100000 0x00c000 0x00c000 R 0x8
LOAD 0x00c0f0 0xffffffff8010c000 0x000000000010c000 0x004032 0x005002 RWE 0x10
GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RWE 0x8
Section to Segment mapping:
Segment Sections...
00 .boot
01 .text .text._ZN2io3outEth .data .rodata .bss
02
The ALIGN of my text section is 0x10 here, whereas it is 0x8 when everything works properly.
Why is this align only present in some cases ? e.g. when I use this kind of C++ static initialization :
struct interrupt_desc
{
uint16_t clbk_low = 0x0;
uint16_t selector = 0x08;
uint8_t zero = 0x0;
uint8_t flags = 0xE;
uint16_t clbk_mid = 0x0;
uint32_t clbk_high = 0x0;
uint32_t zero2 = 0x0;
} __attribute__((packed));
interrupt_desc bar[32];
or when I implement these assembly structures for IDT handling, if I put them in the .data section, everything if fine, but if they go to the .text, the align is there again :
[SECTION .data]
[GLOBAL _interrupt_table_register]
[GLOBAL _interrupt_vector_table]
[BITS 64]
align 8
_interrupt_table_register:
DW 0xABCD
DQ _interrupt_vector_table
_interrupt_vector_table:
%rep 256
DW 0x0000
DW 0x0008
DB 0x00
DB 0x0E
DW 0x0000
DD 0x00000000
DD 0x00000000
%endrep
And finally, I don't really understand what this align value means and how I can adjust it. Moreover the load address is already 0x10 bytes-aligned, so why does this align changes the effective load address ?
Maybe I fail to understand something important here so any explanation about the internals of ELF is very welcome.
In case anyone wonders, the binary is an ELF64 linked with GNU ld, booted by Grub2 using Multiboot2.
PS: there is nothing wrong with my virtual memory mappings as they work perfectly when the align is 8. also, the shift is present event when I dump physical memory.
also, previously, I used to use Rust for the main code instead of C++. With Rust, this align bug was always present since the implementation of LTO in the compiler, but fine previously.
Thanks for your answers.

LLVM NVPTX backend struct parameter zero size

I'm getting an obscure exception when loading the PTX assembly generated by LLVM's NVPTX backend. (I'm loading the PTX from ManagedCuda - http://managedcuda.codeplex.com/ )
ErrorNoBinaryForGPU: This indicates that there is no kernel image available that is suitable for the device. This can occur when a user specifies code generation options for a particular CUDA source file that do not include the corresponding device configuration.
Here is the LLVM IR for the module (it's a bit weird since it's generated by a tool)
; ModuleID = 'Module'
target triple = "nvptx64-nvidia-cuda"
%testStruct = type { i32 }
define void #kernel(i32 addrspace(1)*) {
entry:
%1 = alloca %testStruct
store %testStruct zeroinitializer, %testStruct* %1
%2 = load %testStruct* %1
call void #structtest(%testStruct %2)
ret void
}
define void #structtest(%testStruct) {
entry:
ret void
}
!nvvm.annotations = !{!0}
!0 = metadata !{void (i32 addrspace(1)*)* #kernel, metadata !"kernel", i32 1}
and here is the resulting PTX
//
// Generated by LLVM NVPTX Back-End
//
.version 3.1
.target sm_20
.address_size 64
// .globl kernel
.visible .func structtest
(
.param .b0 structtest_param_0
)
;
.visible .entry kernel(
.param .u64 kernel_param_0
)
{
.local .align 8 .b8 __local_depot0[8];
.reg .b64 %SP;
.reg .b64 %SPL;
.reg .s32 %r<2>;
.reg .s64 %rl<2>;
mov.u64 %rl1, __local_depot0;
cvta.local.u64 %SP, %rl1;
mov.u32 %r1, 0;
st.u32 [%SP+0], %r1;
// Callseq Start 0
{
.reg .b32 temp_param_reg;
// <end>}
.param .align 4 .b8 param0[4];
st.param.b32 [param0+0], %r1;
call.uni
structtest,
(
param0
);
//{
}// Callseq End 0
ret;
}
// .globl structtest
.visible .func structtest(
.param .b0 structtest_param_0
)
{
ret;
}
I have no idea how to read PTX, but I have a feeling the problem has to do with the .b0 bit of .param .b0 structtest_param_0 in the structtest function definition.
Passing non-structure values (like integers or pointers) works fine, and the .b0. bit of the function reads something sane like .b32 or .b64 when doing so.
Changing triple to nvptx-nvidia-cuda (32 bit) does nothing, as well as including/excluding the data layout suggested in http://llvm.org/docs/NVPTXUsage.html
Is this a bug in the NVPTX backend, or am I doing something wrong?
Update:
I'm looking through this - http://llvm.org/docs/doxygen/html/NVPTXAsmPrinter_8cpp_source.html - and it appears as if the type is falling through to line 01568, is obviously not a primitive type, and Ty->getPrimitiveSizeInBits() returns zero. (At least that's my guess, anyway)
Do I need to add a special case for checking to see if it's a structure, taking the address, making the argument byval, and dereferencing the struct afterwards? That seems like a hacky solution, but I'm not sure how else to fix it.
Have you tried to get the error message buffer from compilation? In managedCuda this would be something like:
CudaContext ctx = new CudaContext();
CudaJitOptionCollection options = new CudaJitOptionCollection();
CudaJOErrorLogBuffer err = new CudaJOErrorLogBuffer(1024);
options.Add(err);
try
{
ctx.LoadModulePTX("test.ptx", options);
}
catch
{
options.UpdateValues();
MessageBox.Show(err.Value);
}
When I run your ptx it says:
ptxas application ptx input, line 12; fatal : Parsing error near '.b0': syntax error
ptxas fatal : Ptx assembly aborted due to errors"
what supports your guess with b0.