In my application i have 2 LLVM modules - the runtime one (which contains void foo(int * a) function definition) and executable one (which i'm creating using LLVM C++ API).
In my executable module i create int main(int argc, char ** argv) and want to put llvm::CallInst into it's body, which would call foo() function from runtime module.
Here is my code:
Function * fooF = Function::Create(runtimeModule->getFunction("foo")->getFunctionType(),
GlobalValue::WeakAnyLinkage, "foo", execModule);
After that, i link two modules together:
Linker linker("blabla", execModule, false);
linker.LinkInFile("/path/to/runtime.bc", false);
execModule = linker.releaseModule();
This compiles OK, however when i run Verifier pass on linked module i get:
Global is external, but doesn't have external or dllimport or weak linkage!
void (%i32*)* #foo
invalid linkage type for function declaration
void (%i32*)* #foo
It's worth mentioning, that all globals in runtime module are internalized using Internalize pass. After linking, but before running Verifier, i'm running Dead Global Elimination pass amongst some other optimizations. And when i do dump() on resulting module, i see, that #foo which is coming from runtime module gets removed too, despite it's used by main(). It seems, LLVM thinks that #foo definition in runtime and #foo declaration in executable are unrelated.
I've tried to play with linkage types - no luck.
So, what is the right way to create a call to the function from another module?
Ok, i've fixed it, but i still can't understand what was the problem. During building of my runtime bitcode module, i've been applying internalize transformation on it. So i tried to do this at run-time after linking and it helped me.
Ah, and i've been using GlobalValue::WeakAnyLinkage.
Related
In my project, I am emitting LLVM IR which makes calls to external functions in dynamic libraries.
I declare my external functions like:
declare %"my_type"* #"my_function"()
In the external library, functions are declared like:
extern "C" {
my_type* my_function();
}
When I compile the IR and run it, the process immediately crashes. The same behavior happens if I declare and call a nonsense function that I know doesn't exist, so I assume what's happening is that the external function is not being found/linked. (I don't think that the function itself is crashing).
I am using Python's llvmlite library for this task, and within the same process where I JIT and invoke my LLVM IR, I have another python library imported which requires the external dynamic library; so I assume that library is loaded and in-memory.
The procedure I'm using to compile and execute my LLVM code is basically the same as what's in this document, except that the IR declares and invokes an external function. I have tried invoking cos(), as in the Kaleidoscope tutorial, and this succeeds, so I am not sure what is different about my own library functions.
I have tried adding an underscore to the beginning of the function name, but I get the same result. (Do I need to add the underscore in the LLVM function declaration?)
How do I verify my assumption that the process is crashing because the named function isn't found?
How do I diagnose why the function isn't being found?
What do I need to do in order to make use of external functions in a dynamic library, from LLVM code?
Edit: It seems there is indeed trouble getting the function pointers to my external function. If I try to merely print the function address by replacing my calls with %"foo" = ptrtoint %"my_type"* ()* #"my_function" to i64 and return/print the result, it still segfaults. Merely trying to obtain the pointer is enough to cause a crash! Why is this, and how do I fix it?
Edit: Also forgot to mention— this is on Ubuntu (in a Docker container, on OSX).
I figured it out— I was missing that I need to call llvmlite.binding.load_library_permanently(filename) in order for the external symbols to become available. Even though the library is already in memory, the function still needs to be invoked. (This corresponds to the LLVM native function llvm::sys::DynamicLibrary::LoadLibraryPermanently()).
From this answer, it appears calling the above function with nullptr will import all the symbols available to the process.
Oddly, on OSX, I found that the external symbols were available even though load_library_permanently() had not been explicitly called— I'm not sure why this is (perhaps the OSX build of llvmlite itself happened to call the function with nullptr, as above?).
I have come across something similar to this piece of code today.
In A.h:
class A { blah blah blah }
#define CREATE_A_FUNC_NAME ("CreateA")
extern "C" A* CreateA(void);
typedef A* (*funcCreateA)(void);
In main.cpp:
void* handle = dlopen("libA.so", RTLD_LAZY);
funcCreateA func = (funcCreateA)dlsym(handle, CREATE_A_FUNC_NAME);
A* a = func();
Now obviously A.h is only the header for declarations and all its implementations are stored in libA.so.
I have tested that if I set up my project correctly, meaning the lib is correctly linked, I can simply do A* a = CreateA() to get the pointer to a newly created A instance. Hence here come the questions. Why go through so much hassle to achieve something simple as one function call? What is this kind of technology or technique called? What are the pros and cons? When should I use this technique? Thanks!
The main reasons to use dlsym rather than linking to the DSO directly:
you want to provide a plugin mechanism with your app, so you need to be able to load a DSO on the fly (The plugins aren't known to the linker when you built the exe). The simplest approach for this is to add some virtual base class (I'm assuming 'A' has some virtual methods?), and then export a creator method with "C" linkage (which disables C++ name mangling). It looks as though that's the intent here?
You may have code optimised for a specific CPU instruction set (i.e. at startup of a game engine, check which is the latest instruction set the CPU supports, load the relevant SSE or AVX library at runtime, and then call the methods optimised for that particular CPU).
In rare cases you might want to 'unload' some heavy code to free up more memory on the device. This happens quite a lot on Android/iOS and consoles (e.g. releasing the shader compiler after all the shaders have been compiled).
It's worth noting that if you link to the DSO directly, under the hood, the linker will simply insert dlsym/dlopen code at app start up, which will automatically load the DSO and resolve the symbols.
AVR g++ has a pointer size of 16 bits. However, my particular chip (the ATMega2560) has 256 KB of RAM. To support this, the compiler automatically generates trampoline sections in the same section of ROM as the current executing code that then contains the extended assembly code to jump into high memory or back. In order for trampolines to be generated, you must take the address of something that sits in high memory.
In my scenario, I have a bootloader that I have written sitting in high memory. The application code needs to be able to call a function in the bootloader. I know the address of this function and need to be able to directly address it by hard-coding the address in my code.
How can I get the compiler/linker to generate the appropriate trampoline for an arbitrary address?
Compiler and linker will only generate trampoline code when the far address is a symbolic address rather than a literal constant number already in code. something like (assuming the address you want to jump to is 0x20000).
extern void (*farfun)() = 0x20000;
farfun ();
Will definitely not work, it doesn't cause the linker to do anything because the address is already resolved.
You should be able to inject the symbol address in the linker command line like so:
extern void farfun ();
farfun ();
compiling "normally" and linking with
-Wl,--defsym,farfun=0x20000
I think it's clear that you need to make sure yourself that something sensible sits at farfun.
You will most probably also need --relax.
EDIT
Never tried this myself, but maybe:
You could probably try to store the function address in a table in high memory and declare it like this:
extern void (*farfunctable [10])();
(farfunctable [0])();
and use the very same linker command to resolve the external symbol (now your table at 0x20000 (in the bootloader) needs to look like this:
extern void func1();
extern void func2();
void ((*farfunctab [10])() = {
func1,
func2,....
};
I would recommend to put func1() ... func10() in a different module from farfunctab in order to make the linker know it has to generate trampolins.
I was planning on putting a dispatch struct (that is, a struct with function pointers to all the various functions). Your solution works well, but requires knowing all of the locations of all of the functions ahead of time. Is there a way to execute a function call to a far address that isn't known at compile time?
[...] My goal was to put the struct with pointers to the functions in a fixed location. That way, it would be a single thing that needed a fixed address rather than every external function.
So you have two applications, let's call them App and Boot, where Boot provides some functionalities that App wants to use. The following problems have to be addressed:
How to get addresses from Boot into App.
How to build a jump table for Boot.
Avoid constructs that will crash when App tries to use code from Boot, like: Using indirect calls or jumps, using static constructors or using static storage in Boot.
App uses Addresses of boot.elf directly
Linking with -Wl,-R,boot.elf
A simple way would be to just link app.elf against boot.elf be means of -Wl,-R,boot.elf. Option -R instructs the linker to use symbol values from the specified file without dragging any code. Problem is that there's no way to specify which symbols to use, for example this might lead to a situation where App uses libgcc functions from Boot.
Defining Symbols by means of -Wl,--defsym,symbol=value
A bit more control over which symbols are being defined can be implemented by following a specific naming convention. Suppose that all symbols from Boot that have "boot" in their name should be "exported", then you could just
> avr-nm -g boot.elf | grep ' T ' | awk '/boot/ { printf("--defsym %s=0x%s\n",$3,$1) }' > syms.opt
This prints global symbol values, and grep filters out symbols in the text section. awk then transforms lines like 00020102 T boot1 to lines like
--defsym boot1=0x00020102 which are written to an option file syms.opt. The option file can then be provided to the linker by means of -Wl,#syms.opt.
The advantage of an option file is that it is easier to provide than plain options in a build environment like make: app.elf would depend (amongst others) on syms.opt, which in turn would depend on boot.elf.
Defining Symbols in a Linker Script Snippet
An alternative would be to define the symbols in a linker script augmentation, which you would provide by means of -T syms.ld during link and which would contain
"boot1"=ABSOLUTE(0x00020102);
"boot2"=...
...
INSERT AFTER .text
Defining Symbols in an Assembly Module
Yet another way to define the symbols would be by means of an assembly module which contains definitions like .global boot1 together with boot1 = 0x00020102.
All these approaches have in common that all symbols must be defined, or otherwise the linker will throw an undefined symbol error. This means boot.elf must be available, and it does not matter whether just one symbol is undefined or whether dozends of symbols are undefined.
Let Boot provide a Dispatch Table
The problem with using boot.elf directly, like lined out in the previous section, is that it introduces a direct dependency. This means that if Boot is improved or refactored, then you'll also have to re-compile App each time, even if the interface did not change.
A solution is to let Boot provide a dispatch table whose position and layout are known ahead of time. Only when the interface itself changes, App will have to be rebuilt. Just refactoring Boot will not require to re-build App.
The Assembly Module with the Jump Table
As explained in the "Crash" section below, addresses in a dispatch table (and hence indirect jumps) won't work because EIND has a wrong value. Therefore, let's assume we have a table of JMPs to the desired Boot functions, like in an assembly module boot-table.sx that reads:
;;; Linker description file boot.ld locates input section .boot.table
;;; right after .vectors, hence the address of .boot_table will be
;;; text-section-start + _VECTORS_SIZE, where the latter is
;;; #define'd in <avr/io.h>.
;;; No "x" section flag so that the linker won't relax JMPs to RJMPs.
.section .boot.table,"a",#progbits
.global .boot_table
.type .boot_table,#object
boot_table:
jmp boot1
jmp boot2
.size boot_table, .-boot_table
In this example, we are going to locate the jump table right after .vectors, so that its location is known ahead of time. The respective symbol definitions in App's syms.opt will then read
--defsym boot1=0x20000+vectors_size+0*4
--defsym boot2=0x20000+vectors_size+1*4
provided Boot is located at 0x20000. Symbol vectors_size can be defined in a C/C++ module, here by abusing avr-gcc attribute "address":
#include <avr/io.h>
__attribute__((__address__(_VECTORS_SIZE)))
char vectors_size;
Locating the Jump Table
In order to locate input section .boot.table, we need an own linker description file, which you might already use for Boot anyways. We start with a linker script from avr-gcc installation at ./avr/lib/ldscripts/avr6.xn, copy it to boot.ld, and add the following 2 lines after vectors:
...
.text :
{
*(.vectors)
KEEP(*(.vectors))
*(.boot.table)
KEEP(*(.boot.table))
/* For data that needs to reside in the lower 64k of progmem. */
*(.progmem.gcc*)
...
Auto-Generating Boot's Jump Table Module and the Symbols for App
It's highly advisable to have an interface description used by both App and Boot, say common.h. Moreover, in order to keep Boot's boot-table.sx and App's syms.opt in sync with the interface, it's agood idea to auto-generate these two files from common.h. To that end, assume that common.h reads:
#ifndef COMMON_H
#define COMMON_H
#define EX __attribute__((__used__,__externally_visible__))
EX int boot1 /* #boot_table:0 */ (int);
EX int boot2 /* #boot_table:1 */ (void);
#endif /* COMMON_H */
For the matter of simplicity, let's assume that this is C code or the interfaces are extern "C" so that the symbols in source code match the assembly names, and there's no need to use mangled names. It' easy enough to generate boot-table.sx and syms.opt from common.h using the magic comments. The magic comment follows directly after the symbol, so a regex would retrieve the token left of the magic comment, something like Python:
# ... symbol /* #boot_table:index */...
pat = re.compile (r".*(\b\w+\b)\s*/\* #boot_table:(\d+) \*/.*")
for line in sys.stdin.readlines():
match = re.match (pat, line)
if match:
index = int (match.group(2))
symbol = match.group(1)
Output template for syms.opt would be something like:
asm_line = "--defsym {symbol}=0x20000+vectors_size+4*{index}\n"
Code that will crash
Using Boot code from App is subject to several restrictions:
Indirect Calls and Jumps
These will crash because the start addresses of App resp. Boot are in different 128KiB segments of flash. When the address of a code symbol is taken, the compiler does this per gs(symbol) which instructs the linker to generate a stub and resolve gs() to that stub in .trampolines if the target address is outside the 128KiB segment where the trampolines are located. An explanation of gs() can be found in this answer, there is however more to it: The startup code will effectively initialize
EIND = __vectors >> 17;
see gcrt1.S, the AVR-LibC bits of start-up code crt<device>.o. The compiler assumes EIND never changes during execution, see EIND and more than 128KiB of Flash in the GCC documentation.
This means code in Boot assumes EIND = 1 but is called with EIND = 0 and hence EICALL resp. EIJMP will target the wrong address. This means common code must avoid indirect calls and jumps, and should be compiled with -fno-jump-tables so that switch/case won't generate such tables.
This also implies that the dispatch table described above won't work if it would just held gs(symbol) entries, because App and Boot will disagree on EIND.
Data in Static Storage
If common Boot code is using data in static storage, the data might collide with App's static storage. One way out is to avoid static storage in respective parts of Boot and pass addresses to, say, some data buffer by means of pointer erguments of respective functions.
One could have completely separate RAM areas; one for Boot and one for App, but that would be a waste of RAM because the applications will never run at the same time.
Static Constructors
Boot's static constructors will be bypassed if App uses code from Boot. This includes:
C++ code in Boot that explicitly or implicitly generates such constructors.
C/C++ code in Boot that relies on __attribute__((__constructor__)) or code in section .initN which is supposed to run prior to main.
Start-up code that initializes static storage, EIND etc., which is also run by locating it in some .initN sections, but will be bypassed if App calls Boot code.
I'm currently working on a modular project and some of our systems won't work, I'll try to explain what we're trying to do.
We've got a main project that is extended by a number of modules (dlls), these modules can have bootstrapper code that will register itself before the main loop starts. It will register itself using our bootstrapping system, however the problem for us is that the modules will not register itself unless we specifically call a function from said modules. We think this is because the linker removes unreferenced code as part of optimizations (this also happens in debug mode).
The main function is set up as follows:
#include <bootstrap/bootstrapper.hpp>
#include <module/module.hpp>
int main()
{
// foo is an empty function in the module header file
foo(); // if I were to remove this empty function,
// the modules bootstrapping code will not execute
return fade::bootstrap::run();
}
without the foo() function the bootstrapping code doesn't get executed
void foo()
{
}
namespace
{
std::unique_ptr<game> game_;
FADE_BOOTSTRAP_MODULE(module_game) // registers itself to the bootstrapper
}
We've tried a number of things such as the linker options:
/INCLUDE
/OPT:NOREF
/EXPORT
But to no avail, either it gives us undefined symbol errors or it does nothing at all.
Is there anything we can do so that the unreferenced code doesn't get optimized away? We want to keep our project modular and cross platform so we'd rather not hardcode any solutions to our main function.
I want to do application, which can be compiled with external modules, for example like in php. In php you can load modules in runtime, or compile php with modules together, so modules are available without loading in runtime. But i don't understand how this can be done. If i have module in module.c and there is one function, called say_hello, how can i register it to main application, if you understand what i mean?
/* module.c */
#include <stdio.h>
// here register say_hello function, but how, if i can't in global scope
// call another function?
void say_hello()
{
printf("hello!");
}
If i compile all that files(main app + modules) together, there isn't some reference to say_hello function from main app, because it is called only if user call it in its code. So how can i say to my app, hey, there is say_hello function, if someone want to call it, you know it exists.
EDIT1: I need to have something like table at runtime, where i can see if user called function exists (have C equivavent). Header files doesn't help to me.
EDIT2: My app is interpret for my script langugage.
EDIT3: If someone call function in php, php interpret must know that function exists. I know about dynamic linking and if .so or .dll is loaded, then some start routine is called and you can simple register function in that dll, so php interpret can see, if some module registred for example function called "say_hello". But if i want compile php with for example gd support, then how gd functions are registred to some php function list, hashtable or whatever?
I guess what you are looking for is dynamic libraries (we call runtime loadable modules as dynamic/shared libraries in C and in the OS world, in general). Take, for example, Pidgin which supports plugins to extend it's functionalities. It gives a particular interface to it's plugin-makers to abide by, say functions to register, load, unload and use, which the plugins will have to follow.
When the program loads, it looks for such dynamic libraries in it's plugins directory, if present, it'll load and use it, else it'll skip exposing the functionality. The reason why an interface is needed is that since different modules can have different functionalities which are unknown uptil runtime, an app. has to have a common, agreed-upon way of "talking" to it's plugins/modules.
Every C program can be linked to a static or a dynamic library; static will copy the code from the library to the said program, there by leaving no dependencies for the program to run, while linking to a dynamic library expects the dynamic library to be present when the program is launched. A third way of doing it, is not to link to a DLL, but just asking the OS to perform a load operation of the library. If this succeeds, then the dynamic module is used, else ignored. The functionality which the dynamic library should perform is exposed to the user, only if the load call succeeds.
It is to be noted that this is a operating system provided feature and it has nothing to do with the language used (C or C++ or Python doesn't matter here); as far as C is concered, the compiler still links to known code i.e. code which is available # compile time. This is the reason for different operating system, one needs to write different code to load a dynamic module. Even more, the file type/format of syuch libraries vary from system to system. In Linux it's called shared objects (.so), in Mac it's called dynamic libraries (.dylib) and in Windows as Dynamic link libraries (.dll).
C is not interpreted language. So you need linking, you may want static linking or dynamic linking.
Program building consists of 2 major phases: compiling and linking. During compiling all c-files are translated into machine code, leaving called functions unresolved (obj or o files). Then linker merges all these files into one executable, resolving what was unresolved.
This is static linking. Linked module becomes integral part of executable.
Dynamic linking is platform specific. Under windows these are DLLs. You should issue a system call to load DLL after which you will be able to call functions from it.
What you need is dynamic library. Let's first take a look at the example provided in the Linux manpage of dlopen(3):
/* Load the math library, and print the cosine of 2.0: */
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
int main(int argc, char **argv) {
void *handle;
double (*cosine)(double);
char *error;
handle = dlopen("libm.so", RTLD_LAZY);
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
dlerror(); /* Clear any existing error */
/* Writing: cosine = (double (*)(double)) dlsym(handle, "cos");
would seem more natural, but the C99 standard leaves
casting from "void *" to a function pointer undefined.
The assignment used below is a workaround. */
*(void **) (&cosine) = dlsym(handle, "cos");
if ((error = dlerror()) != NULL) {
fprintf(stderr, "%s\n", error);
exit(EXIT_FAILURE);
}
printf("%f\n", (*cosine)(2.0));
dlclose(handle);
exit(EXIT_SUCCESS);
}
There's also a C++ dlopen mini HOWTO.
For more general information about dynamic loading, start from the wikipedia page first.
I think it is impossible, if i understand what you mean. Because it is compiled language.