I've been following Apple's Dynamic Library Programming Topics
docs to create and use a runtime-loaded library using dlopen() / dlsym().
It seems I'm getting a failure to find the desired symbol on my Mid 2012 MacBook Air, running macOS Mojave.
Library Source Code
// adder.h
int add(int x);
and
// adder.cpp
#include "adder.h"
int add(int x) {
return (x + 1);
}
Compiled with clang -dynamiclib adder.cpp -o libAdd.A.dylib
Main Source
// main.cpp
#include <stdio.h>
#include <dlfcn.h>
#include <stdlib.h>
#include "adder.h"
int main() {
void* adder_handle = dlopen("libAdd.A.dylib", RTLD_LOCAL|RTLD_LAZY);
if (!adder_handle) {
printf("[%s] Unable to load library: %s\n\n", __FILE__, dlerror());
exit(EXIT_FAILURE);
}
while(true) {
void* voidptr = dlsym(adder_handle, "add");
int (*add)(int) = (int (*)(int))voidptr;
if (!add) {
printf("[%s] Unable to get symbol: %s\n\n", __FILE__, dlerror());
exit(EXIT_FAILURE);
}
printf("%d\n", add(0));
}
dlclose(adder_handle);
return 0;
}
Compiled with clang main.cpp -o main
I've also set the DYLD_LIBRARY_PATH environment variable to ensure the library can be found. Everything compiles ok.
Nevertheless, when I run the main executable, I get the error:
[main.cpp] Unable to get symbol: dlsym(0x7fb180500000, add): symbol not found
Running nm -gC libAdd.A.dylib outputs:
0000000000000fa0 T add(int)
U dyld_stub_binder
Any ideas on what could be wrong, or what I need to do to debug this issue?
Thanks!
C++ actually mangles the functionname which results in a different symbolname.
Your are able to spot these mangled symbol names using nm -g <yourlib.dylib>
You can change this behavior by wrapping your method into
extern "C" {
int add(int x);
}
Related
I'm writing a program (macOS, clang++ compiler, only AppleSilicon at the moment) that I can extend later by providing custom plugins (dynamic library, loaded at runtime) which use main program's public interface.
test.hpp - public interface:
#if defined(MAGIC_PLUGIN)
# define MAGIC_EXPORT /* nothing */
#else
# define MAGIC_EXPORT __attribute__((visibility("default")))
#endif
MAGIC_EXPORT
void testCall();
test.cpp - main programm:
#include <stdio.h>
#include <dlfcn.h>
#include "test.hpp"
// Declare a function to call from a loaded plugin
typedef void (* plugin_call_func)(void);
int main(int argc, char** argv) {
// Load library
const char* libraryName = "plugin.dylib";
void* library = dlopen(libraryName, RTLD_NOW);
if (library == nullptr) {
printf("Cannot open library\n");
return 1;
}
// Get function from loaded library
plugin_call_func pluginCall = reinterpret_cast<plugin_call_func>(
dlsym(library, "pluginCall"));
if (pluginCall == nullptr) {
printf("Cannot find the pluginCall function\n");
return 2;
}
// Execute loaded function
pluginCall();
// Forget function and close library
pluginCall = nullptr;
auto libraryCloseResult = dlclose(library);
if (libraryCloseResult != 0) {
printf("Cannot close library\n");
return 3;
}
return 0;
}
// Public function, should be called from a plugin
void testCall() {
printf("Test call\n");
}
plugin.cpp - plugin's source:
#define MAGIC_PLUGIN
#include <stdio.h>
#include "test.hpp"
__attribute__((visibility("default")))
extern "C" void pluginCall(void) {
printf("Plugin call\n");
testCall();
}
First, I compile main app:
clang++ -std=c++20 -fvisibility=hidden -target arm64-apple-macos12 test.cpp -o test
The nm --defined-only test shows these symbols:
0000000100003ee4 T __Z8testCallv
0000000100000000 T __mh_execute_header
0000000100003dcc t _main
Mangled __Z8testCallv is what I need. Everything looks good so far. But then I try to compile the plugin as dynamic library...
clang++ -std=c++20 -fvisibility=hidden -dynamiclib -g -current_version 0.1 -target arm64-apple-macos12 plugin.cpp -o plugin.dylib
and get this error:
Undefined symbols for architecture arm64:
"testCall()", referenced from:
_pluginCall in plugin-38422c.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Well, it's kind of fair, I can understand this, because the dynamic library does not know that testCall is somewhere implemented. So I want to say it that it does not have to worry about testCall's existence.
I tried to research how to do this, looked up man pages, read tons of stack overflow answers, and what I only found that works is adding these flags to linker:
-Wl,-undefined,dynamic_lookup
It works, the library compiles and the app works as expected. But I don't really want to use dynamic_lookup because it will mark every undefined symbol in the library as resolved, which may lead to some bad consequences. I want to tell the linker only about existence of the main program's public symbols.
What am I missing? Is there any better solution than dynamic_lookup?
Your best bet is to manually do the work that's done by the library loader. That is: populating function pointers. After all, the plugin->main binding is already done manually, so doing the same thing the other way around makes sense.
You can make this process essentially transparent by carefully crafting the header shared by the plugin and main application. The only tricky part is handling ODR for plugins that are composed of multiple source files.
Since this is a C++ question, here's what it could look like with a RAII wrapper. The ODR conundrum is handled via the PLUGIN_MAIN macro that should only be defined in one of a plugin's sources.
test_plugin.hpp
using pluginCall_fn = void(*)();
using testCall_fn = void(*)();
#if !defined(MAIN_APPLICATION)
#if defined(PLUGIN_MAIN)
#define EXPORTED_FROM_MAIN __attribute__((visibility("default")))
#else
#define EXPORTED_FROM_MAIN __attribute__((visibility("default"))) extern
#endif
extern "C" {
// Declare symbols provided by the plugin
__attribute__((visibility("default"))) void pluginCall();
// etc...
// Declare/define pointers that will be populated by the main application
EXPORTED_FROM_MAIN testCall_fn testCall;
// etc...
}
#undef EXPORTED_FROM_MAIN
#else // In the main app.
#include <stdexcept>
// Declare "symbols" provided by the main application
void testCall();
// Utility class to load/unload a dynamic library.
// Probably belongs in its own header...
struct loaded_library final {
loaded_library(const char* libName)
: handle_(dlopen(libName, RTLD_NOW)) {
if(!handle_) {
throw std::runtime_error("failed to load plugin");
}
}
loaded_library(const loaded_library&) = delete;
loaded_library& operator=(const loaded_library&) = delete;
loaded_library(loaded_library&& rhs) : handle_(rhs.handle_) {
rhs.handle_ = nullptr;
}
loaded_library& operator=(loaded_library&& rhs) {
handle_ = rhs.handle_;
rhs.handle_ = nullptr;
return *this;
}
~loaded_library() {
if(handle_) {
dlclose(handle_);
}
}
template<typename T>
T get_symbol(const char* symbol) {
T result = reinterpret_cast<T>(dlsym(handle_, symbol));
if(!result) {
throw std::runtime_error("missing symbol");
}
return result;
}
private:
void* handle_;
};
// Plugin interface.
struct loaded_plugin final {
loaded_plugin(const char* libName)
: lib_(libName) {
// Load functions from plugin
pluginCall = lib_.get_symbol<pluginCall_fn>("pluginCall");
// ...
// Assign callbacks to plugin
*lib_.get_symbol<testCall_fn*>("testCall") = &testCall;
// ...
// Call the plugin's init function here if applicable.
}
pluginCall_fn pluginCall;
private:
loaded_library lib_;
};
#endif
plugin.cpp
#define PLUGIN_MAIN
#include "test_plugin.hpp"
#include <stdio.h>
void pluginCall() {
printf("Plugin call\n");
testCall();
}
test.cpp
#define MAIN_APPLICATION
#include "test_plugin.hpp"
int main(int argc, char** argv) {
const char* libraryName = "plugin.dylib";
loaded_plugin plugin(libraryName);
plugin.pluginCall();
}
// Public function, should be called from a plugin
void testCall() {
printf("Test call\n");
}
You may find that this code is a bit on the fragile side of things, since a few different portions of test_plugin.hpp need to be kept in sync.
This can be worked around with the use of X-Macros, at the cost of confusing IDEs and hurting code legibility. I wouldn't go down that road until the APIs in question become unwieldingly large.
I am new to SWIG and I am trying some tutorials but running into compilation issues.
The functions I am trying to wrap (ex.cxx) :
#include <time.h>
double My_variable = 3.0;
int fact(int n) {
if (n <= 1) return 1;
else return n*fact(n-1);
}
int my_mod(int x, int y) {
return (x%y);
}
char *get_time()
{
time_t ltime;
time(<ime);
return ctime(<ime);
}
the SWIG wrapper definition (ex.i) :
%module ex
%{
/* Put header files here or function declarations like below */
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
%}
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
the minimal setup to call the wrappers (min.cxx) :
#include <stdio.h>
extern "C" {
#include <lua.h>
#include <lualib.h>
#include <lauxlib.h>
}
#include <string.h>
//#include "ex_wrap.cxx"
extern "C" {
extern int luaopen_ex(lua_State* L); // declare the wrapped module
}
int main(int argc,char* argv[]) {
char buff[256];
int error;
lua_State *L;
if (argc<2) {
printf("%s: <filename.lua>\n",argv[0]);
return 0;
}
L=luaL_newstate(); // https://stackoverflow.com/questions/8552560/embedding-lua-in-c
luaL_openlibs(L); // load basic libs (eg. print)
luaopen_ex(L); // load the wrappered module
if (luaL_loadfile(L,argv[1])==0) // load and run the file
lua_pcall(L,0,0,0);
else
printf("unable to load %s\n",argv[1]);
/*while (fgets(buff, sizeof(buff), stdin) != NULL) {
error = luaL_loadbuffer(L, buff, strlen(buff), "line") ||
lua_pcall(L, 0, 0, 0);
if (error) {
fprintf(stderr, "%s", lua_tostring(L, -1));
lua_pop(L, 1); // pop error message from the stack
}
}*/
lua_close(L);
return 0;
}
and the compile commands :
swig -debug-symtabs -debug-symbols -debug-csymbols -o ex_wrap.cxx -c++ -lua ex.i
clang -std=c++11 -I/usr/local/include/lua -c min.cxx -o min.o
clang -std=c++11 -fvisibility=default -I/usr/local/include/lua -c ex_wrap.cxx -o ex_wrap.o
clang -std=c++11 -c ex.cxx -o ex.o
clang -std=c++11 -I/usr/local/include/lua -L/usr/local/lib/lua -llua ex_wrap.o min.o ex.o -o mylua
clang -shared -std=c++11 -I/usr/local/include/lua -L/usr/local/lib/lua -llua min.o ex.o -o ex.so
The last command fails with
Undefined symbols for architecture x86_64:
"_luaopen_ex", referenced from:
_main in min.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
luaopen_ex() is defined in the ex_wrap.cxx generated by the swig command, but the linker cannot seem to find it. BTW, if I include ex_wrap.cxx directly into min.cxx, it compiles and I can run Lua code with this mini interpreter afterwards. Any ideas, thank you.
luaopen_example is defined in an extern "C" {} block in ex_wrap.cxx, but compiling ex.c with g++ declares luaopen_example as a C++ function, so the linker looks for a C++ mangled name to resolve luaopen_example(lua_State*) and not simply luaopen_example
Change your declaration in ex.c to
extern "C" {
extern int luaopen_example(lua_State* L); // declare the wrapped module
}
and the code will compile.
( You might be also be interested in the answer to this question, which explains name mangling: What is the effect of extern "C" in C++? )
it's a linking failure error, and it happens because it can't link luaopen_example, but why it happens? luaopen_example is to load the wrapped module as you mentioned and it's named "example" only if the module name is the same.
in swig and lua wrapper, the wrapper name depends on the file to be wrapped name, but if you don't want to go this way, you can use the option -o, then the wrappered module will export one function "int luaopen_example(lua_State* L)" which must be called to register the module with the Lua interpreter. The name "luaopen_example" depends upon the name of the module.
so i suggest you just keep the name of the wrapper derived from the module, or you can adjust the loading function luaopen_xxxx to your wrapper name.
for full resource you can take a look at SWIG and Lua
Setup
Suppose I wrote a program in C/C++ and wanted to allow plugin loading. The typical solution would be to write the plugin as:
plugin.c
int my_plugin_fn() {
return 7;
}
And compile it using something like gcc -fpic -shared -o plugin.so plugin.c
Then, in the main program that loads this plugin, we might have:
loader.c
#include <stdio.h>
#include <dlfcn.h>
int main() {
void *plugin_handle = dlopen("./plugin.so", RTLD_LAZY);
if (!plugin_handle) {
printf("Could not open shared object: %s\n", dlerror());
return -1;
}
int (*fn)() = dlsym(plugin_handle, "my_plugin_fn");
char *err = dlerror();
if (err) {
printf("Could not resolve symbol: %s\n", err);
return -1;
}
printf("Plugin object returned: %d\n", fn());
return 0;
}
I compiled loader.c with gcc -o loader loader.c -ldl, and after running it, the output was Plugin object returned: 7, as expected.
Question
Suppose we want to add functions in our main program (loader.c) that plugins can use. For example,
loader_v2.c
#include <stdio.h>
#include <dlfcn.h>
int times_2(int x) {
return 2*x;
}
int main() {
void *plugin_handle = dlopen("./plugin_v2.so", RTLD_LAZY);
if (!plugin_handle) {
printf("Could not open shared object: %s\n", dlerror());
return -1;
}
int (*fn)() = dlsym(plugin_handle, "my_plugin_fn");
char *err = dlerror();
if (err) {
printf("Could not resolve symbol: %s\n", err);
return -1;
}
printf("Plugin object returned: %d\n", fn());
return 0;
}
plugin_v2.c
extern int times_2(int);
int my_plugin_fn() {
return times_2(7);
}
Compiling and running these files in the same way as before produces Could not open shared object: ./loader_v2: symbol lookup error: ./plugin_v2.so: undefined symbol: times_2.
Is there a way to have plugins loaded using dlopen() call functions from the program that loaded them?
Is there a way to have plugins loaded using dlopen() call functions from the program that loaded them?
Yes, but the function that you want to call from the main executable must be exported from it, which does not happen by default. You can see which symbols are exported from your main binary with nm -D loader.
You can export all functions defined in the main executable by linking it with the -rdynamic flag.
Some linkers, most notably newer versions of GNU-ld, GOLD and LLD, support the --export-dynamic-symbol flag, which allows one to selectively export just the symbol(s) you need.
In your case, you would link your loader executable with -Wl,--export-dynamic-symbol=times_2.
The best way to do something like this is with a function pointer. You would pass the function pointer into the library function, which would subsequently call it.
So the library function would look like this:
int my_plugin_fn(int (*cb)(int)) {
return cb(7);
}
The call to dlsym would then look like this:
int (*fn)(int (*)(int)) = dlsym(plugin_handle, "my_plugin_fn");
And you would call the library function like this:
printf("Plugin object returned: %d\n", fn(times_2));
I have two versions of a a shared library:
library version 2:
simple.h
#pragma once
int first(int x);
simple.c
#include "simple.h"
#include <stdio.h>
__asm__(".symver first_1_0,first#LIBSIMPLE_1.0");
int first_1_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return x + 1;
}
__asm__(".symver first_2_0,first##LIBSIMPLE_2.0");
int first_2_0(int x)
{
int y;
printf("lib: %d\n", y);
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * 1000;
}
linker version script file:
LIBSIMPLE_1.0{
global:
first;
local:
*;
};
LIBSIMPLE_2.0{
global:
first;
local:
*;
};
gcc -Wall -g -O0 -fPIC -c simple.c
gcc -shared simple.o -Wl,--version-script,script -o libsimple.so.2.0.0
And library version 3:
simple.h
#pragma once
#ifdef SIMPLELIB_VERSION_3_0
int first(int x, int normfactor);
#else
int first(int x);
#endif //SIMPLELIB_VERSION_3_0
simple.c
#include "simple.h"
#include <stdio.h>
__asm__(".symver first_1_0,first#LIBSIMPLE_1.0");
int first_1_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return x + 1;
}
__asm__(".symver first_2_0,first#LIBSIMPLE_2.0");
int first_2_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * 1000;
}
__asm__(".symver first_3_0,first##LIBSIMPLE_3.0");
int first_3_0(int x, int normfactor)
{
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * normfactor;
}
Linker version script file:
LIBSIMPLE_1.0{
global:
first; second;
local:
*;
};
LIBSIMPLE_2.0{
global:
first;
local:
*;
};
LIBSIMPLE_3.0{
global:
first;
local:
*;
};
gcc -Wall -g -O0 -fPIC -c simple.c
gcc -shared simple.o -Wl,--version-script,script -o libsimple.so.3.0.0
So i end up with having two different libraries. Next i create a simple application, that eventually i want to link to a library version 3, so in it i use function first() that takes two arguments:
main.c
#include <stdio.h>
#include "simple.h"
int main(int argc, char* argv[])
{
int nFirst = first(1, 10);
printf("First(1) = %d\n", nFirst);
}
I compile app with next commands:
gcc -g -Wall -DSIMPLELIB_VERSION_3_0 -c main.c
And then, by accident, instead of linking to a library version 3, i linked against library version 2. I expected linking to fail, but it went through, and application was working.
gcc main.o -Wl,-L. -lsimple.2.0.0 -Wl,-R. -o demo
So my questions are:
Is it because library exports symbol with name 'function', and application tries to link to the same symbol name, and that is why linker didn't complain, and just linked against library version 2 ?
I thought since c++ mangles symbol names, such thing wouldn't happen, and linker wouldn't link to a library version 2. So i tried all the same, but instead of a gcc compiler, i tried to use g++. Everything went well, until i tried to link application to a library, and i received unresolved links error. Can not figure out why.
p.s. Sorry for a big amount of code. I was trying to make it clear.
Thanks
Is it because library exports symbol with name 'function', and application tries to link to the same symbol name, and that is why linker didn't complain, and just linked against library version 2 ?
Yes, since plain C does not have function overload there is no need for mangling and as a consequence only function name will be used as a symbol for linking. In the end your application code wants to link with function and your library code exports function and this is enough to keep linker happy (even though it is not valid from binary interface perspective).
I thought since c++ mangles symbol names, such thing wouldn't happen, and linker wouldn't link to a library version 2. So i tried all the same, but instead of a gcc compiler, i tried to use g++. Everything went well, until i tried to link application to a library, and i received unresolved links error. Can not figure out why.
Yes, this problem should not occur in C++ because of name mangling. However, this is true only if you have both your application code and library code in C++ or if you bridge your C and C++ code the right way.
It is hard to say (without full listing) what happened in your case when you used g++ but from the looks of it you ended up having application code in C++ and library code still in C. If that is the case your application code will now want to link with mangled function while your library code still exports unmangled function.
To verify this you can inspect your object file with something like:
nm main.o
... and see exactly what kind of symbol does it want. If you will get something like this:
...
U _Z3functionii
...
... instead of:
...
U function
...
... then that is the case.
To "fix" this and make your C++ application code link with unmangled function from library code you'll need to declare your function prototype as extern "C".
I've got a variable called global_count that I would like to share between the shared library and the main part of the program (and ultimately other libs, but for now I've simplified the testcase).
Given the declaration of global_count in globals.cpp:
extern "C" {
int* global_count;
}
We compile to create a global.o file:
gcc -c global.cpp
shared.cpp below will be used to create shared.so:
#include <stdio.h>
#include "global.h"
extern "C" {
void init_global_count(int* xp) {
printf("Initialize global count to: %d\n", *xp);
global_count = xp;
};
void print_global_count(){
if(global_count) {
printf("global_count is: %d\n",*global_count);
} else {
printf("global_count* is not initialized!\n");
}
};
}
global.h:
extern "C" {
extern int* global_count;
}
main.cpp:
#include <stdlib.h>
#include <dlfcn.h>
#include <stdio.h>
#include "global.h"
int answer = 42;
int* gc_copy;
typedef void (*func)();
void (*init_global_count)(int*);
void (*print_global_count)();
void load(char* shared_lib){
void* handle;
handle = dlopen(shared_lib,RTLD_NOW | RTLD_GLOBAL) ;
printf("load:after dlopen\n");
if(!handle)
{
printf("DID NOT LOAD!\n");
fflush(stdout);
fputs (dlerror(), stderr);
exit(1);
} else {
printf("Loaded OK!\n");
fflush(stdout);
void (*init_global_count)(int*) = (void (*)(int*))dlsym(handle, "init_global_count");
(*init_global_count)(&answer);
void (*print_global_count)() = (void (*)())dlsym(handle, "print_global_count");
(*print_global_count)();
}
}
int main(){
printf("main...\n");
load((char*)"./shared.so");
if(global_count)
printf("main:global_count is: %d\n", *global_count);
return 0;
}
To compile the shared lib and main:
gcc -g -Wall -fno-omit-frame-pointer -fPIC -shared -o shared.so shared.cpp global.o
gcc -g -o main main.cpp global.o -ldl
Note that we're linking in global.o in both of those compilations.
Now we run it and the output is:
main...
load:after dlopen
Loaded OK!
Initialize global count to: 42
global_count is: 42
So the *global_count* reported from inside *print_global_count()* (defined in shared.cpp) is 42 as expected. However, global_count is not reported from main because the global_count pointer has not been initialized - so the global_count in main.cpp is not the same as the global_count in shared.cpp.
I'm wondering if it's possible to do what I'm trying to do here (to share some global data between a .so and the module that loads the .so)? If so, do I need to link differently?
All objects in a shared library need to be compiled as position independent (-fpic or -fPIC). So linking in globals.o into your shared library is in error.
You are actually creating two instances of the global_count pointer. You are trying to print out a pointer value as a decimal integer from main. The global pointer variable global_count in the main version is not yet initialized, so it has the starting value of all bytes set to 0.
You can remove globals.o from your shared library, and only have one instance of the global_count variable in main. However, for the shared library to see the main global variables, they have to be made visible to them. You can do this by adding -rdynamic to the flags to gcc in your link line for main.