I am working on an embedded system (STM32, ARM M33). I am developing both bootloader and application code. The bootloader and application both use the same filesystem code to access external FLASH memory. Since the size of this code is NOT trivial and it won't change (at least not very often), I would like to have only one copy of it located in the MCU to be a "shared library."
I have referenced the following articles looking for a solution:
Linker script: insert absolute address of the function to the generated code
https://www.embeddedrelated.com/showthread/comp.arch.embedded/213239-1.php
Bootloader and main application to share common code/functionalities
One option is to hard-code addresses to the functions and force the linker (of the bootloader) to place these functions at those addresses. This is very hard to maintain and prone to all sorts of problems.
Option 2 is not much better. It involves exporting a list of symbols from the bootloader and linking the application against this so that my shared functions are linked directly into the bootloader's address space.
Option 3 is to locate some sort of jump table at a very specific address within the bootloader's address space (similar to an interrupt vector). The application code would then call the filesystem functions indirectly via this vector. I think I know how to accomplish something like this using a linker script and a special section in flash.
Finally, one of the articles mentioned "create a jump table or a C++ object
that implements a virtual interface." Since I am using C++ for my application, this seems the most intriguing option to me to use a virtual interface. From my understanding, virtual methods work by two levels of indirection. The object pointer gets you to a vtable, then the vtable gets you to the actual methods. This is very similar to a C-style jump table but with concrete language support.
My question is, how would this be implemented in practice?
At the moment, my bootloader starts executing the application code by calling the Reset ISR from the application's interrupt vector table (the same function the hardware itself would call immediately after reset). In doing this, the bootloader has no way to "pass on" information (i.e., a pointer to a virtual object) to the application.
Your first link is the right thing to do, except you should scrape the ROM map file to generate your linker/symbol definition file that you link to.
The bigger problem is ensuring the symbols you're linking to aren't referencing other symbols that aren't alive any more like static or global variables.
The first option is also the approach that some semiconductor provide ROM code. Normally, the share software should have stable interface since this will mostly unable to change/update in the future. Therefore, it is not necessary to think about the maintenance of share code in the future.
Other option might fit to some special need. However, they might increase the complexity of your software.
pls advise how I can load 64-bit DLLs from memory in C/C++,
i'm running Windows 64-bit mode, I found few solutions for
Win32 but none working correctly with 64-bit
i don't want to create any temporary files, i want to have
all req DLLs into single .bin file and delay-load DLL
into my .exe program
links and example source are most welcome!
thanks
You can load and execute code in memory at runtime, with no temporary files. Strictly speaking, it's not a DLL, just dynamic code.
You will need the same API functions used by programs that generate code at runtime (many languages these days have just-in-time compilers).
In principle, there are only three steps:
Get the machine code into dynamically allocated memory. Whether you read it from a file, transfer it over the network, or generate it using a compiler doesn't matter.
Mark the pages containing the code as executable. On Windows, that's done with VirtualProtect. On Linux, mprotect.
Synchronize the instruction and data cache (modern CPUs are a hybrid of Harvard and von Neumann architectures). On Windows, that's FlushInstructionCache.
The latter two bullets will need the base address and size of your buffer containing the code.
What's more difficult is dealing with code that has dependencies. Since you're not using the OS loader, you have no static import resolution and no fixups. If your code is designed for this situation (accepts a pointer to the host application data as a function argument and goes through that for all external interactions) this doesn't have to be a problem. Since you're thinking in terms of a DLL, though, it likely will be.
Still, this is far from an insurmountable problem. Start by getting a position-independent function working though before you move on to fixups and imports.
I want to know if it is possible to "edit" the code inside an already compiled DLL.
I.E. imagine that there is a function called sum(a,b) inside Math.dll which adds the two numbers a and b
Let's say i've lost the source code of my DLL. So the only thing i have is the binary DLL file.
Is there a way i could open that binary file, locate where my function resides and replace the sum(a,b) routine with, for example, another routine that returns the multiplication of a and b (instead of the sum)?
In Summary, is it posible to edit Binary code files?
maybe using reverse engineering tools like ollydbg?
Yes it is definitely possible (as long as the DLL isn't cryptographically signed), but it is challenging. You can do it with a simple Hex editor, though depending on the size of the DLL you may have to update a lot of sections. Don't try to read the raw binary, but rather run it through a disassembler.
Inside the compiled binary you will see a bunch of esoteric bytes. All of the opcodes that are normally written in assembly as instructions like "call," "jmp," etc. will be translated to the machine architecture dependent byte equivalent. If you use a disassembler, the disassembler will replace these binary values with assembly instructions so that it is much easier to understand what is happening.
Inside the compiled binary you will also see a lot of references to hard coded locations. For example, instead of seeing "call add()" it will be "call 0xFFFFF." The value here is typically a reference to an instruction sitting at a particular offset in the file. Usually this is the first instruction belonging to the function being called. Other times it is stack setup/cleanup code. This varies by compiler.
As long as the instructions you replace are the exact same size as the original instructions, your offsets will still be correct and you won't need to update the rest of the file. However if you change the size of the instructions you replace, you'll need to manually update all references to locations (this is really tedious btw).
Hint: If the instructions you're adding are smaller than what you replaced, you can pad the rest with NOPs to keep the locations from getting off.
Hope that helps, and happy hacking :-)
Detours, a library for instrumenting arbitrary Win32 functions on x86 machines. Detours intercepts Win32 functions by re-writing target function images. The Detours package also contains utilities to attach arbitrary DLLs and data segments (called payloads) to any Win32 binary.
Download
You can, of course, hex-edit the DLL to your heart's content and do all sorts of fancy things. But the question is why go to all that trouble if your intention is to replace the function to begin with?
Create a new DLL with the new function, and change the code that calls the function in the old DLL to call the function in the new DLL.
Or did you lose the source code to the application as well? ;)
I'm currently working on some system level code where I would like to be able to identify the memory section(s) that are from the loaded binary in order to detect things like corrupted or modified instructions;
Essentially what I'm after is a way, in Win32 using C++, to get a pointer to the range of instructions. This is somewhat similar to asking for a function pointer to the .text section's start and end. My understanding of the exe format is that the .text section is where instructions are stored, versus the .data section which holds things like global variables. Unfortunately I've found 0 hints on where this might be (I've seen no win32 function calls, nothing in the TIB, etc.)
Can anyone direct me to where I could find/calculate this information?
P.S. I do understand that if anyone changes code maliciously that they may find this code and change it; I'm still interested in the details of how to get at this information for my own curiosity.
You can't really expect this to work with an in memory binary. Any function calls to imported DLLs will get modified by the loader to point to the actual locations of the target procedures in the DLL that is loaded.
For example suppose you call a function in kernel32.dll. Then a Windows update happens which changes kernel32.dll. The next time you run your app, the jump to the function in kernel32.dll is going to be to a different memory address than the before the Windows update was applied.
And of course this all assumes that DLLs load at their preferred address. And then you may have some self-modifying code.
And so on, and so on.
You can find the entry-point to your code in the PE header. Download the PE (Portable Executable) file definition from MSDN - it has all the information. The format of the program in memory is virtually the same as it is on disk. From within the code, you can get a pointer to the PE header in memory via the GetModuleHandle() function (the handle is really a pointer to the first page).
This doesn't directly answer your question, but for your overall solution, you could look into Code Signing. If you like this solution, there are existing implementations on Windows.
As you said, binary verification alone won't solve your problem. You should also look into installing your application in an area of the file system that requires elevation/admin rights to write to, such as Program Files, or deploy it somewhere a user can't directly modify it, like a web server.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
So I found out that C(++) programs actually don't compile to plain "binary" (I may have gotten some things wrong here, in that case I'm sorry :D) but to a range of things (symbol table, os-related stuff,...) but...
Does assembler "compile" to pure binary? That means no extra stuff besides resources like predefined strings, etc.
If C compiles to something else than plain binary, how can that small assembler bootloader just copy the instructions from the HDD to memory and execute them? I mean if the OS kernel, which is probably written in C, compiles to something different than plain binary - how does the bootloader handle it?
edit: I know that assembler doesn't "compile" because it only has your machine's instruction set - I didn't find a good word for what assembler "assembles" to. If you have one, leave it here as comment and I'll change it.
C typically compiles to assembler, just because that makes life easy for the poor compiler writer.
Assembly code always assembles (not "compiles") to relocatable object code. You can think of this as binary machine code and binary data, but with lots of decoration and metadata. The key parts are:
Code and data appear in named "sections".
Relocatable object files may include definitions of labels, which refer to locations within the sections.
Relocatable object files may include "holes" that are to be filled with the values of labels defined elsewhere. The official name for such a hole is a relocation entry.
For example, if you compile and assemble (but don't link) this program
int main () { printf("Hello, world\n"); }
you are likely to wind up with a relocatable object file with
A text section containing the machine code for main
A label definition for main which points to the beginning of the text section
A rodata (read-only data) section containing the bytes of the string literal "Hello, world\n"
A relocation entry that depends on printf and that points to a "hole" in a call instruction in the middle of a text section.
If you are on a Unix system a relocatable object file is generally called a .o file, as in hello.o, and you can explore the label definitions and uses with a simple tool called nm, and you can get more detailed information from a somewhat more complicated tool called objdump.
I teach a class that covers these topics, and I have students write an assembler and linker, which takes a couple of weeks, but when they've done that most of them have a pretty good handle on relocatable object code. It's not such an easy thing.
Let's take a C program.
When you run gcc, clang, or 'cl' on the c program, it will go through these stages:
Preprocessor (#include, #ifdef, trigraph analysis, encoding translations, comment management, macros...) including lexing into preprocessor tokens and eventually resulting in flat text for input to the compiler proper.
Lexical analysis (producing tokens and lexical errors).
Syntactical analysis (producing a parse tree and syntactical errors).
Semantic analysis (producing a symbol table, scoping information and scoping/typing errors) Also data-flow, transforming the program logic into an "intermediate representation" that the optimizer can work with. (Often an SSA). clang/LLVM uses LLVM-IR, gcc uses GIMPLE then RTL.
Optimization of the program logic, including constant propagation, inlining, hoisting invariants out of loops, auto-vectorization, and many many other things. (Most of the code for a widely-used modern compiler is optimization passes.) Transforming through intermediate representations is just part of how some compilers work, making it impossible / meaningless to "disable all optimizations"
Outputing into assembly source (or another intermediate format like .NET IL bytecode)
Assembling of the assembly into some binary object format.
Linking of the assembly into whatever static libraries are needed, as well as relocating it if needed.
Output of final executable in elf, PE/coff, MachO64, or whatever other format
In practice, some of these steps may be done at the same time, but this is the logical order. Most compilers have options to stop after any given step (e.g. preprocess or asm), including dumping internal representation between optimization passes for open-source compilers like GCC. (-ftree-dump-...)
Note that there's a 'container' of elf or coff format around the actual executable binary, unless it's a DOS .com executable
You will find that a book on compilers(I recommend the Dragon book, the standard introductory book in the field) will have all the information you need and more.
As Marco commented, linking and loading is a large area and the Dragon book more or less stops at the output of the executable binary. To actually go from there to running on an operating system is a decently complex process, which Levine in Linkers and Loaders covers.
I've wiki'd this answer to let people tweak any errors/add information.
There are different phases in translating C++ into a binary executable. The language specification does not explicitly state the translation phases. However, I will describe the common translation phases.
Source C++ To Assembly or Itermediate Language
Some compilers actually translate the C++ code into an assembly language or an intermediate language. This is not a required phase, but helpful in debugging and optimizations.
Assembly To Object Code
The next common step is to translate Assembly language into an Object code. The object code contains assembly code with relative addresses and open references to external subroutines (methods or functions). In general, the translator puts in as much information into an object file as it can, everything else is unresolved.
Linking Object Code(s)
The linking phase combines one or more object codes, resolves references and eliminates duplicate subroutines. The final output is an executable file. This file contains information for the operating system and relative addresses.
Executing Binary Files
The Operating System loads the executable file, usually from a hard drive, and places it into memory. The OS may convert relative addresses into physical locations. The OS may also prepare resources (such as DLLs and GUI widgets) that are required by the executable (which may be stated in the Executable file).
Compiling Directly To Binary
Some compilers, such as the ones used in Embedded Systems, have the capability to compile from C++ directly to an executable binary code. This code will have physical addresses instead of relative address and not require an OS to load.
Advantages
One of the advantages of these phases is that C++ programs can be broken into pieces, compiled individually and linked at a later time. They can even be linked with pieces from other developers (a.k.a. libraries). This allows developers to only compiler pieces in development and link in pieces that are already validated. In general, the translation from C++ to object is the time consuming part of the process. Also, a person doesn't want to wait for all the phases to complete when there is an error in the source code.
Keep an open mind and always expect the Third Alternative (Option).
To answer your questions, please note that this is subjective as there are different processors, different platforms, different assemblers and C compilers, in this case, I will talk about the Intel x86 platform.
Assemblers do not usually assemble to pure / flat binary (raw machine code), instead usually to a file defined with segments such as data, text and bss to name but a few; this is called an object file. The Linker steps in and adjusts the segments to make it executable, that is, ready to run. Incidentally, the default output when you assemble using GNU as foo.s is a.out, that is a shorthand for Assembler Output. (But the same filename is the gcc default for linker output, with the assembler output being only a temporary.)
Boot loaders have a special directive defined, back in the days of DOS, it would be common to find a directive such as .Org 100h, which defines the assembler code to be of the old .COM variety before .EXE took over in popularity. Also, you did not need to have a assembler to produce a .COM file, using the old debug.exe that came with MSDOS, did the trick for small simple programs, the .COM files did not need a linker and were straight ready-to-run binary format. Here's a simple session using DEBUG.
1:*a 0100
2:* mov AH,07
3:* int 21
4:* cmp AL,00
5:* jnz 010c
6:* mov AH,07
7:* int 21
8:* mov AH,4C
9:* int 21
10:*
11:*r CX
12:*10
13:*n respond.com
14:*w
15:*q
This produces a ready-to-run .COM program called 'respond.com' that waits for a keystroke and not echo it to the screen. Notice, the beginning, the usage of 'a 100h' which shows that the Instruction pointer starts at 100h which is the feature of a .COM. This old script was mainly used in batch files waiting for a response and not echo it. The original script can be found here.
Again, in the case of boot loaders, they are converted to a binary format, there was a program that used to come with DOS, called EXE2BIN. That was the job of converting the raw object code into a format that can be copied on to a bootable disk for booting. Remember no linker is run against the assembled code, as the linker is for the runtime environment and sets up the code to make it runnable and executable.
The BIOS when booting, expects code to be at segment:offset, 0x7c00, if my memory serves me correct, the code (after being EXE2BIN'd), will start executing, then the bootloader relocates itself lower down in memory and continue loading by issuing int 0x13 to read from the disk, switch on the A20 gate, enable the DMA, switch onto protected mode as the BIOS is in 16bit mode, then the data read from the disk is loaded into memory, then the bootloader issues a far jump into the data code (likely to be written in C). That is in essence how the system boots.
Ok, the previous paragraph sounds abstracted and simple, I may have missed out something, but that is how it is in a nutshell.
To answer the assembly part of the question, assembly doesn't compile to binary as I understand it. Assembly === binary. It directly translates. Each assembly operation has a binary string that directly matches it. Each operation has a binary code, and each register variable has a binary address.
That is, unless Assembler != Assembly and I'm misunderstanding your question.
They compile to a file in a specific format (COFF for Windows, etc), composed of headers and segments, some of which have "plain binary" op codes. Assemblers and compilers (such as C) create the same sort of output. Some formats, such as the old *.COM files, had no headers, but still had certain assumptions (such as where in memory it would get loaded or how big it could be).
On Windows machines, the OS's boostrapper is in a disk sector loaded by the BIOS, where both of these are "plain". Once the OS has loaded its loader, it can read files that have headers and segments.
Does that help?
There are two things that you may mix here. Generally there are two topics:
Executable File Formats (see a list here), for example COFF, XCOFF, ELF
Intermediate Languages, like CIL or GIMPLE or bytecode
The latter may compile to the former in the process of assembly. Some intermediate formats are not assembled, but executed by a virtual machine. In case of C++ it may be compiled into CIL, which is assembled into a .NET assembly, hence there me be some confusion.
But in general C and C++ are usually compiled into binary, or in other words, into a executable file format.
You have a lot of answers to read through, but I think I can keep this succinct.
"Binary code" refers to the bits that feed through the microprocessor's circuits. The microprocessor loads each instruction from memory in sequence, doing whatever they say. Different processor families have different formats for instructions: x86, ARM, PowerPC, etc. You point the processor at the instruction you want by giving it the address of the instruction in memory, and then it chugs merrily along through the rest of the program.
When you want to load a program into the processor, you first have to make the binary code accessible in memory so it has an address in the first place. The C compiler outputs a file in the filesystem, which has to be loaded into a new virtual address space. Therefore, in addition to binary code, that file has to include the information that it has binary code, and what its address space should look like.
A bootloader has different requirements, so its file format might be different. But the idea is the same: binary code is always a payload in a larger file format, which includes at a minimum a sanity check to ensure that it's written in the correct instruction set.
C compilers and assemblers are typically configured to produce static library files. For embedded applications, you're more likely to find a compiler which produces something like a raw memory image with instructions beginning at address zero. Otherwise, you can write a linker which converts the output of the C compiler into whatever else you want.
As I understand it, a chipset (CPU, etc.) will have a set of registers for storing data, and understand a set of instructions for manipulating these registers. The instructions will be things like 'store this value to this register', 'move this value', or 'compare these two values'. These instructions are often expressed in short human-grokable alphabetic codes (assembly language, or assembler) which are mapped to the numbers that the chipset understands - those numbers are presented to the chip in binary (machine code.)
Those codes are the lowest level that the software gets down to. Going deeper than that gets into the architecture of the actual chip, which is something I haven't gotten involved in.
The executable files (PE format on windows) cannot be used to boot the computer because the PE loader is not in memory.
The way bootstrapping works is that the master boot record on the disk contains a blob of a few hundred bytes of code. The BIOS of the computer (in ROM on the motherboard) loads this blob into memory and sets the CPU instruction pointer to the beginning of this boot code.
The boot code then loads a "second stage" loader, on Windows called NTLDR (no extension) from the root directory. This is raw machine code that, like the MBR loader, is loaded into memory cold and executed.
NTLDR has the full capability to load PE files including DLLs and drivers.
ะก(++) (unmanaged) really compiles to plain binary. Some OS-related stuff - are BIOS and OS function calls, they're different for each OS, but still binary.
1. Assembler compiles to pure binary, but, as strange as it gets, it is less optimized than C(++)
2. OS kernel, as well as bootloader, also written in C, so no problems here.
Java, Managed C++, and other .NET stuff, compiles into some pseudocode (MSIL in .NET), which makes it cross-OS and cross-platform, but requires local interpreter or translator to run.