I'm experimenting with making a kind of plugin architecture for a program I wrote, and at my first attempt I'm having a problem. Is it possible to access symbols from the main executable from within the shared object? I thought the following would be fine:
testlib.cpp:
void foo();
void bar() __attribute__((constructor));
void bar(){ foo(); }
testexe.cpp:
#include <iostream>
#include <dlfcn.h>
using namespace std;
void foo()
{
cout << "dynamic library loaded" << endl;
}
int main()
{
cout << "attempting to load" << endl;
void* ret = dlopen("./testlib.so", RTLD_LAZY);
if(ret == NULL)
cout << "fail: " << dlerror() << endl;
else
cout << "success" << endl;
return 0;
}
Compiled with:
g++ -fPIC -o testexe testexe.cpp -ldl
g++ --shared -fPIC -o testlib.so testlib.cpp
Output:
attempting to load
fail: ./testlib.so: undefined symbol: _Z3foov
So obviously, it's not fine. So I guess I have two questions:
1) Is there a way to make the shared object find symbols in the executable it's loaded from
2) If not, how do programs that use plugins typically work that they manage to get code in arbitrary shared objects to run inside their programs?
Try:
g++ -fPIC -rdynamic -o testexe testexe.cpp -ldl
Without the -rdynamic (or something equivalent, like -Wl,--export-dynamic), symbols from the application itself will not be available for dynamic linking.
From The Linux Programming Interface:
42.1.6 Accessing Symbols in the Main Program
Suppose that we use dlopen() to dynamically load a shared library, use dlsym() to obtain
the address of a function x() from that library, and then call x(). If
x() in turn calls a function y(), then y() would normally be sought in
one of the shared libraries loaded by the program. Sometimes, it is
desirable instead to have x() invoke an implementation of y() in the
main program. (This is similar to a callback mechanism.) In order to
do this, we must make the (global-scope) symbols in the main program
available to the dynamic linker, by linking the program using the
––export–dynamic linker option:
$ gcc -Wl,--export-dynamic main.c (plus further options and arguments)
Equivalently, we can write the
following:
$ gcc -export-dynamic main.c
Using either of these options
allows a dynamically loaded library to access global symbols in the
main program.
The gcc –rdynamic option and the gcc –Wl,–E option are
further synonyms for –Wl,––export–dynamic.
Related
In a main program, I dlopen and dlclose (LoadLibrary and FreeLibrary respectively) a shared library. The shared library contains a static variable that is instantiated upon dlopen, and destroyed upon dlclose. This behavior is consistent on MSVC 2008 and 2013, GCC 3.4.6, and Sunstudio 12.1.
With GCC 4.9.1 and GCC 5.2.1 however, the destructor was no longer called on dlclose. Instead, it was called before program exit.
The particularity of the static variable's class, is that in the constructor, there is a call to a templated function get (of global scope) that returns a local static variable.
I was able to reproduce this behavior with the following one cpp file linked into a shared library:
#include <iostream>
template <typename T> // In my actual code, i is of type T, however, this has no effect
int get()
{
static int i = 0;
return i;
}
class Dictionary {
public:
Dictionary()
{
std::cout << "Calling Constructor" << std::endl;
get<int>();
}
~Dictionary(){
std::cout << "Calling Destructor" << std::endl;
}
private:
Dictionary(const Dictionary&);
Dictionary& operator=(const Dictionary&);
};
static Dictionary d;
I investigated the tweaks that can be made in order to have the destructor called on dlclose, and concluded the following:
If the function get was not templated
else if the variable i in the function get was not static
else if the function get is made static
The main program's code is the following:
#include <dlfcn.h>
#include <cassert>
#include <string>
#include <iostream>
void* LoadLib(std::string name)
{
void* libInstance;
name = "lib" + name + ".so";
libInstance = dlopen(name.c_str(), RTLD_NOW);
if ( ! libInstance ) std::cout << "Loading of dictionary library failed. Reason: " << dlerror() << std::endl;
return libInstance;
}
bool UnloadLib(void* libInstance)
{
int ret = dlclose(libInstance);
if (ret == -1)
{
std::cout << "Unloading of dictionary library failed. Reason: " << dlerror() << std::endl;
return false;
}
return true;
}
int main()
{
void* instance = LoadLib("dll");
assert(instance != 0);
assert(UnloadLib(instance));
std::cout << "DLL unloaded" << std::endl;
}
I built the binaries with the following commands:
g++ -m64 -g -std=c++11 -shared -fPIC dll.cpp -o libdll.so
g++ -m64 -g -std=c++11 -ldl main.cpp -o main.out
The output I get when the destructor is called before program exit is the following:
Calling Constructor
DLL unloaded
Calling Destructor
The output I get when the destructor is called on dlclose is the following:
Calling Constructor
Calling Destructor
DLL unloaded
Questions:
If the change of behavior between the versions of GCC is not a bug, can you please explain why is the destructor not called on dlclose?
Can you please explain for each tweak: Why is the destructor called on dlclose in this case?
There is no guarantee that unloading (destructors are invoked) happens on dlclose. On musl (as opposed to glibc), constructors only run the first time a library is run, and destructors only run on exit. For portable code, dlclose cannot be assumed to unload the symbols immediately.
The unload behavior depends on glibc's symbol binding when doing dynamic linking, and is independent of GCC.
The static variable get::i has a STB_GNU_UNIQUE binding. For static variables in inline functions, the uniqueness of the object is assured by the ELF linker. However, for dynamic loading, the dynamic linker assures uniqueness by marking the symbol STB_GNU_UNIQUE. Hence, another attempt to dlopen the same shared library by some other code will lookup the symbol and find that it is unique and return the existent one from the unique symbols table. A symbol with unique binding cannot be unloaded.
Unique binding can be disabled with -fno-gnu-unique if not needed.
References
Bug that I raised to GCC
STB_GNU_UNIQUE
I've been working on a couple established C++ projects that use static variables from a shared library to store parameters. When compiled with g++ or clang++, the static variable is shared (has the same memory location) throughout the entire program. However, when compiled with Xcode, the main function static variable has a different memory location than the shared library static variable. Is there a way to get Xcode to compile/run the code the same as g++ or clang++, while still being able to debug with Xcode?
Please see example below:
main.cpp:
#include <iostream>
#include "Params.hpp"
int main(int argc, const char * argv[]) {
Params param = Params();
param.addParams();
std::vector<int> vi = Params::ParamsObj();
vi.push_back(10);
for(std::vector<int>::iterator it = vi.begin(); it != vi.end(); ++it) {
std::cout << "i = " << *it << std::endl;
}
return 0;
}
Params.hpp:
#ifndef Params_hpp
#define Params_hpp
#include <vector>
class Params{
typedef std::vector<int> ParamVector;
public:
static ParamVector& ParamsObj() {
static ParamVector m;
return m;
}
void addParams();
};
#endif /* Params_hpp */
Params.cpp:
#include "Params.hpp"
void Params::addParams(){
Params::ParamsObj().push_back(5);
}
Makefile:
clang:
clang++ -dynamiclib Params.cpp -o libshared_clang.dylib
clang++ main.cpp -o main_clang ./libshared_clang.dylib
gpp:
g++-mp-4.9 -Wall -shared -fPIC -o libshared_gpp.so Params.cpp
g++-mp-4.9 -Wall -o main_gpp main.cpp ./libshared_gpp.so
Output from both g++ and clang++ is:
i = 5
i = 10
While Xcode only outputs i = 10.
If I don't use a shared library and compile everything into one binary, Xcode will properly output both print statements.
My current solution is to add the project's main function into its own shared library and then create an Xcode specific file which merely calls the main function in the newly created shared library. However, I was hoping for a solution that didn't require changing the underlying project's code.
I'm pretty sure that if you turn on optimalization for gcc/clang (which you did not in your example), they will produce the same behavior as your compilation with XCode (which isn't a compiler, but an IDE).
Your problem is that the ParamsObj() function is inline (defining it in the class body adds an implicit inline keyword to it), allowing the compiler to just "paste" it into the main method instead of calling it.
With dll boundaries, this might result in the allocation of multiple static variables, if the function is used in multiple libraries (in your case, it's used in the dll, and inlined into the main executable).
Refactor the ParamsObj() method into a declaration and a separate definition in the corresponding C++ file, and you'll get the same behavior everywhere, printing both numbers.
This question is closely related to dlopen a dynamic library from a static library linux C++, but contains a further complication (and uses C++ instead of C):
I have an application that links against a static library (.a) and that library uses the dlopen function to load dynamic libraries (.so). In addition, the dynamic libraries call functions defined in the static one.
Is there a way to compile this without linking the dynamic libraries against the static one or vice versa?
Here comes what I tried so far, slightly modifying the example from the related question:
app.cpp:
#include "staticlib.hpp"
#include <iostream>
int main()
{
std::cout << "and the magic number is: " << doSomethingDynamicish() << std::endl;
return 0;
}
staticlib.hpp:
#ifndef __STATICLIB_H__
#define __STATICLIB_H__
int doSomethingDynamicish();
int doSomethingBoring();
#endif
staticlib.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingDynamicish()
{
void* handle = dlopen("./libdynlib.so",RTLD_NOW);
if(!handle)
{
std::cout << "could not dlopen: " << dlerror() << std::endl;
return 0;
}
typedef int(*dynamicfnc)();
dynamicfnc func = (dynamicfnc)dlsym(handle,"GetMeANumber");
const char* err = dlerror();
if(err)
{
std::cout << "could not dlsym: " <<err << std::endl;
return 0;
}
return func();
}
staticlib2.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingBoring()
{
std::cout << "This function is so boring." << std::endl;
return 0;
}
dynlib.cpp:
#include "staticlib.hpp"
extern "C" int GetMeANumber()
{
doSomethingBoring();
return 1337;
}
and build:
g++ -c -o staticlib.o staticlib.cpp
g++ -c -o staticlib2.o staticlib2.cpp
ar rv libstaticlib.a staticlib.o staticlib2.o
ranlib libstaticlib.a
g++ -rdynamic -o app app.cpp libstaticlib.a -ldl
g++ -fPIC -shared -o libdynlib.so dynlib.cpp
When I run it with ./app I get
could not dlopen: ./libdynlib.so: undefined symbol: _Z17doSomethingBoringv
and the magic number is: 0
From the dlopen manual page:
If the executable was linked with the flag "-rdynamic" (or, synonymously, "--export-dynamic"), then the global symbols in the executable will also be used to resolve references in a dynamically loaded library.
That means that for the application to export its symbols for use in the dynamic library, you have to link your application with the -rdynamic flag.
Besides the problem described above, there is another problem and that has to do with the static library: The problem is namely that since the doSomethingBoring function is not called in your main program, the object file staticlib2.o from the static library is not linked.
The answer can be found in e.g. this old question, which tells you to add the --whole-archive linker flag:
g++ -rdynamic -o app app.cpp -L. \
-Wl,--whole-archive -lstaticlib \
-Wl,--no-whole-archive -ldl
Is it possible to compile (C++) code for the GPU with nvcc into a shared object (.so file) and load it dynamically from a C++ program (in this case, Cern's ROOT, which is essentially a C++ interpreter ("CINT")).
A simple example that I would like to run is:
extern "C"
void TestCompiled() {
printf("test\n");
exit(0);
}
This code was compiled with nvcc --compiler-options '-fPIC' -o TestCompiled_C.so --shared TestCompiled.cu. Loading the shared object into ROOT with:
{ // Test.C program
int error, check;
check = gROOT->LoadMacro("TestCompiled_C.so", &error);
cout << "check " << check << " " << " error: " << error << endl;
TestCompiled(); // run macro
exit(0);
}
loads the library OK, but does not find TestCompiled():
$ root -b -l Test.C
root [0]
Processing Test.C...
check 0 error: 0
Error: Function Hello() is not defined in current scope Test.C:11:
*** Interpreter error recovered ***
Doing the same by compiling the first test script with ROOT (without the extern line, compiling with root TestCompiled.C++) works… What can I try in order to make the C++ program find the test function when nvcc does the compilation?
I am assuming that the shared object file being output is like any other shared library, such as one created with GCC using the shared option. In this case, to load the object dynamically, you will need to use the dlopen function to get a handle to the shared object. Then, you can use the dlsym function to look for a symbol in the file.
void *object_handle = dlopen("TestCompiled_C.so", RTLD_NOW);
if (object_handle == NULL)
{
printf("%s\n", dlerror());
// Exit or return error code
}
void *test_compiled_ptr = dlsym(object_handle, "TestCompiled");
if (!test_compiled)
{
printf("%s\n", dlerror());
// Exit or return error code
}
void (*test_compiled)() = (void (*)()) test_compiled_ptr;
test_compiled();
You will need to include dlfcn.h and link with -ldl when you compile.
The difference between this and what you are doing now is that you are loading the library statically rather that dynamically. Even though shared objects are "dynamically linked libraries," as they are called in the windows world, doing it the way you are now is loading all of the symbols in the object when the program is launched. To dynamically load certain symbols at runtime, you need to do it this way.
I'm copying, for reference, the salient points of the answer from the RootTalk forum that solved the problem:
A key point is that the C interpreter of ROOT (CINT) requires a "CINT dictionary" for the externally compiled function. (There is no problem when compiling through ROOT, because ACLiC creates this dictionary when it pre-compiles the macro [root TestCompiled.C++]).
So, an interface TestCompiled.h++ must be created:
#ifdef __cplusplus
extern "C" {
#endif
void TestCompiled(void);
#ifdef __cplusplus
} /* end of extern "C" */
#endif
The interface must then be loaded inside ROOT along with the shared object:
{ // Test.C ROOT/CINT unnamed macro (interpreted)
Int_t check, error;
check = gROOT->LoadMacro("TestCompiled_C.so", &error);
std::cout << "_C.so check " << check << " error " << error << std::endl;
check = gROOT->LoadMacro("TestCompiled.h++", &error);
std::cout << "_h.so check " << check << " error " << error << std::endl;
TestCompiled(); // execute the compiled function
}
ROOT can now use the externally compiled program: root -b -l -n -q Test.C works.
This can be tested with, e.g., g++ on the following TestCompiled.C:
#include <cstdio>
extern "C" void TestCompiled(void) { printf("test\n"); }
compiled with
g++ -fPIC -shared -o TestCompiled_C.so TestCompiled.C
Since I observed some strange behavior of global variables in my dynamically loaded libraries, I wrote the following test.
At first we need a statically linked library: The header test.hpp
#ifndef __BASE_HPP
#define __BASE_HPP
#include <iostream>
class test {
private:
int value;
public:
test(int value) : value(value) {
std::cout << "test::test(int) : value = " << value << std::endl;
}
~test() {
std::cout << "test::~test() : value = " << value << std::endl;
}
int get_value() const { return value; }
void set_value(int new_value) { value = new_value; }
};
extern test global_test;
#endif // __BASE_HPP
and the source test.cpp
#include "base.hpp"
test global_test = test(1);
Then I wrote a dynamically loaded library: library.cpp
#include "base.hpp"
extern "C" {
test* get_global_test() { return &global_test; }
}
and a client program loading this library: client.cpp
#include <iostream>
#include <dlfcn.h>
#include "base.hpp"
typedef test* get_global_test_t();
int main() {
global_test.set_value(2); // global_test from libbase.a
std::cout << "client: " << global_test.get_value() << std::endl;
void* handle = dlopen("./liblibrary.so", RTLD_LAZY);
if (handle == NULL) {
std::cout << dlerror() << std::endl;
return 1;
}
get_global_test_t* get_global_test = NULL;
void* func = dlsym(handle, "get_global_test");
if (func == NULL) {
std::cout << dlerror() << std::endl;
return 1;
} else get_global_test = reinterpret_cast<get_global_test_t*>(func);
test* t = get_global_test(); // global_test from liblibrary.so
std::cout << "liblibrary.so: " << t->get_value() << std::endl;
std::cout << "client: " << global_test.get_value() << std::endl;
dlclose(handle);
return 0;
}
Now I compile the statically loaded library with
g++ -Wall -g -c base.cpp
ar rcs libbase.a base.o
the dynamically loaded library
g++ -Wall -g -fPIC -shared library.cpp libbase.a -o liblibrary.so
and the client
g++ -Wall -g -ldl client.cpp libbase.a -o client
Now I observe: The client and the dynamically loaded library possess a different version of the variable global_test. But in my project I'm using cmake. The build script looks like this:
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)
PROJECT(globaltest)
ADD_LIBRARY(base STATIC base.cpp)
ADD_LIBRARY(library MODULE library.cpp)
TARGET_LINK_LIBRARIES(library base)
ADD_EXECUTABLE(client client.cpp)
TARGET_LINK_LIBRARIES(client base dl)
analyzing the created makefiles I found that cmake builds the client with
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
This ends up in a slightly different but fatal behavior: The global_test of the client and the dynamically loaded library are the same but will be destroyed two times at the end of the program.
Am I using cmake in a wrong way? Is it possible that the client and the dynamically loaded library use the same global_test but without this double destruction problem?
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
CMake adds -rdynamic option allowing loaded library to resolve symbols in the loading executable... So you can see that this is what you don't want. Without this option it just misses this symbol by accident.
But... You should not do any stuff like that there. Your libraries and executable should
not share symbols unless they are really should be shared.
Always think of dynamic linking as static linking.
If using shared libraries you must define the stuff you want to export with macro like here. See DLL_PUBLIC macro definition in there.
By default, the linker won't combine a global variable (a 'D') in the base executable with one in a shared library. The base executable is special. There might be an obscure way to do this with one of those obscure control files that ld reads, but I sort of doubt it.
--export-dynamic will cause a.out 'D' symbols to be available to shared libs.
However, consider the process. Step 1: you create a DSO from a .o with a 'U' and a .a with a 'D'. So, the linker incorporates the symbol in the DSO. Step 2, you create the executable with a 'U' in one of the .o files, and 'D' in both a .a and the DSO. It will try to resolve using the left-to-right rule.
Variables, as opposed to functions, pose certain difficulties for the linker across modules in any case. A better practice is to avoid global var references across module boundaries, and use function calls. However, that would still fail for you if you put the same function in both the base executable and a shared lib.
My first question is if there is any particular reason for which you both statically and dynamically (via dlopen) link the same code?
For your problem: -rdynamic will export the symbols from your program and what probably is happening is that dynamic linker resolves all references to your global variable to the first symbol it encounters in symbol tables. Which one is that I don't know.
EDIT: given your purpose I would link your program that way:
g++ -Wall -g -ldl client.cpp -llibrary -L. -o client
You may need to fix the order.
I would advise to use a dlopen(... RTLD_LAZY|RTLD_GLOBAL); to merge global symbol tables.
I would propose to compile any .a static library which you plan to link to a dinamic library, with -fvisibility=hidden parameter, so:
g++ -Wall -fvisibility=hidden -g -c base.cpp