Dynamically loaded library and global variables in c++ - c++

I'm trying to build a system, that uses dynamic loaded library files (.so files) as plugins in C++14. I build the project using gcc in combination with qt5.5.1 within qtcreator.
The problem I'm having is, that I don't fully understand, what dlopen() (and dlsym()) actually does and get strange behavior because of it. Here is a simplified (not executable) Version:
/*Kernel.hpp*/
class Kernel{
int loadPlugins();
}
void* sharedPointer; //The Object location is stored in here
/*Kernel.cpp*/
Kernel::loadPlugins(){
handle1 = dlopen(<file1>, RTLD_LAZY);
init_t init = (init_t) dlsym(handle1, "init"); //init_t is just a fitting function pointer
execute_t exec = (execute_t) dlsym(handle1, "execute"); //same goes for "execute_t"
handle2 = dlopen(<file2>, RTLD_LAZY);
init_t init = (init_t) dlsym(handle2, "init");
execute_t exec = (execute_t) dlsym(handle2, "execute");
}
/*<file1.h>*/
Class Test{
int func();
int field = 0;
}
/*<file1.cpp>*/
int Test::func(){/*do stuff*/}
Test* test = NULL;
extern void* sharedPtr; //use the ptr from kernel
extern "C" init(){
test = new Test();
sharedPtr = (void*)test; //store address of newly created Test-Object
}
extern "C" execute(){
/* Do Stuff */
}
/*<file2.h>*/
/*<file2.cpp>*/
Test* test = NULL;
extern void* sharedPtr; //use the ptr from kernel
extern "C" init(){
test = (Test*)sharedPtr; //get address of Testobject
}
extern "C" execute(){
std::cout << test->field << std::endl; //Working perfectly
std::cout << test->func() << std::endl //Segmentaion fault
}
The precise Error is a symbol lookup error of a member function with some mangled name (unmangled name is Kernel::test()).
What I think, that should happen:
When Kernel.loadPlugins() is called, the first library creates an object, saves its address in the main program. Library reads that address and can use it, as if it had created that object. So the field can be read and written to and the member function can be called.
What actually happens:
When Kernel.loadPlugins() is called, the first library creates said object, can use it as expected, saves its address in the main program. Library receives said address as expected, can use the field of said object as expected (it does not matter what type that field has, even other objects, like strings, worked, valgrind does not show any leaks as well), but when it tries to call func() it produces a segmentation fault.
I have two primary doubts -
First, I would like to know why that happens?
Second, I would like to know if there is a nice way to fix it?

Related

DLL – static vector which is filled up at DLL’s initialization time, returns zero size to the client program

I am experiencing the following issue, in my DLL project:
At the DLL side :
Inside the DLL I have declared a static vector as follows :
static std::vector<FilterProcessor::FilterInfo*> TableOfContents;
At DLL’s initialization time of static members, I am adding some entries to the above vector.
I have defined an extern “C” global function (getTocPointer()) which is returning a pointer to the vector, when it called from the client program.
extern "C" __declspec(dllexport) std::vector<FilterProcessor::FilterInfo*>* __cdecl getLibraryTOC();
At the client’s program side :
The DLL library is loaded without any problem
The address of getTocPointer() function is returned correctly to the client program, when the getProcAddress() function is called.
Indeed, when I am performing the debugging process in the DLL-side, the client program calls the above function and the execution process enters to it.
However, the vector has a zero size and, has no any contents which were added to it at initialization time. It seems it points to another vector object. . .
I can’t really understand what exactly goes wrong here.
The way of adding entries to this vector at initialization time, is the proper way?
If yes, what probably goes wrong when the client program calls the getLibraryTOC() function?
Thanks in advance
George
If that static global definition of the vector appears in a header file, then yes you do have multiple different vectors. Change the keyword static to extern to make the header file declare the vector rather than defining it, and then add exactly one definition in an implementation file.
Then, you may encounter the static initialization order fiasco. If the vector is defined in a different compilation unit than the code attempting to add entries to it, there's no guarantee that the vector object is alive yet. Attempting to use a vector whose constructor hasn't run is undefined behavior -- it might easily manifest as the constructor running afterward and setting the contents to zero length (as a default constructor should), but many other problems are possible.
You can avoid the SIOF by using a local static.
std::vector<FilterProcessor::FilterInfo*>& table_of_contents()
{
static std::vector<FilterProcessor::FilterInfo*> singleton;
return singleton;
}
In every location that would have accessed the global, including the initialization logic that fills the vector, and also your getLibraryTOC() exported function, call the accessor function instead.
That all is applicable to any C++ software having multiple compilation units. When you have a DLL, things get even more complicated, because the DLL and EXE are compiled and linked separately from each other, possibly with different settings, different compilers, or even entirely different languages. Sharing of complex objects across DLL boundaries is real trouble. If the DLL and EXE are always recompiled at the same time, it can work. But if you're trying to distribute the DLL for use by another party who writes the EXE code, the strong coupling will quickly become intolerable.
A better approach is to hide the library objects from the DLL boundary, and pass only primitive or OS-managed types across. For example:
#define DLLAPI __declspec(dllexport) __cdecl
extern "C" DLLAPI int32_t getLibraryTocCount()
{ return table_of_contents.size(); }
extern "C" DLLAPI BSTR getLibraryTocName(int index)
{ return ::SysAllocString(table_of_contents[index].name.c_str(); } // assumes std::wstring
// etc
The library I have implemented contains the following code (in a brief description) :
An Index class which implements the Table of contents of the library
A collection of audio filters named Filter01, Filter02 etc.
Index.h
struct LIB_SPECS Library_TableOfContents
{
static bool addTOCEntry(FilterInfo* Filter_Info); // add an entry to the TOC
static std::vector<FilterInfo*> TableOfContents; // TOC
};
/*-------------------------------------------------------------------
Called from the client program to return the pointer to TOC */
extern "C" LIB_SPECS std::vector<FlterInfo*>* __cdecl getLibraryTOC();
Index.cpp
/* Define / Initialize static variables */
std::vector<FilterInfo*> Library_TableOfContents::TableOfContents = {};
//=====================================================================
bool Library_TableOfContents::addTOCEntry(FilterInfo* Filter_Info)
{
Library_TableOfContents::TableOfContents.push_back(Filter_Info);
return false;
}
//======================================================================
std::vector<FilterInfo*>* getLibraryTOC()
{
return &Library_TableOfContents::TableOfContents;
}
For each Audio Filter in the library :
Filterxx.h
class LIB_SPECS Filterxx
{
public:
static struct FilterInfo
{
public:
std::string filterName;
std::string filterDescription;
// other filter info
FilterInfo(); // FilterInfo constructor
} Filter_Info;
virtual String doSomeWork(int AvatarId);
virtual void deleteFilter() = 0;
};
Filterxx.cpp
Filterxx::FilterInfo Filterxx::Filter_Info("Filterxx", “A filter description e.g. Low pass Filter ” ); //
FilterInfo::FilterInfo(std::string name, std::string description)
{
Filter_Info.filterName = name;
Filter_Info.filterDescription = description;
Library_TableOfContents::addTOCEntry(&Filter_Info);
}
// other filter functions
The getLibraryTOC() function, is called from the client program to get the table of contents in order to show it to the user.
As I said, indeed it is called by the client but, at the time of call, the table of contents seems to have a zero size.

C++ load shared library and extract class implementations at runtime on linux platform

In C++, is it possible to load a shared library at execution time?
I want the user to choose which shared library to be loaded at runtime, without recompiling the whole program.
dlopen() is a solution for C, but my program is written is C++/Qt, and the symbol to extract are Qt-style class, is there a more "c++" way to do that.
You can do it in Qt using QLibrary in two ways. The following example calls a function from a shared library at runtime in two different ways:
#include <QLibrary>
#include <QDebug>
class Dynamic_library
{
public:
Dynamic_library();
virtual int sum( int len, int * data );
};
typedef Dynamic_library * (*get_object_func)();
typedef int (*call_sum_func)(int len , int * data);
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
QLibrary library( "./dynamic_library" );
library.load();
if( !library.isLoaded() )
{
qDebug() << "Cannot load library.";
return 0;
}
call_sum_func call_sum = (call_sum_func)library.resolve( "call_sum" );
if( call_sum )
{
//Dynamic_library * obj=get_object();
int * a=new int[3];
a[0]=2;
a[1]=3;
a[2]=4;
qDebug() << "sum of 2+3+4' = " << call_sum( 3, a ) <<"\n";
delete [] a;
}
get_object_func get_object = (get_object_func)library.resolve( "get_object" );
if( get_object )
{
Dynamic_library * obj=get_object();
int * a=new int[3];
a[0]=7;
a[1]=8;
a[2]=9;
qDebug() << "sum of 7+8+9' = " << obj->sum(3, a );
delete [] a;
}
return a.exec();
}
The code for the shared library is as follows:
class DYNAMIC_LIBRARYSHARED_EXPORT Dynamic_library
{
public:
Dynamic_library();
virtual int sum( int len, int * data );
};
extern "C" Q_DECL_EXPORT Dynamic_library * get_object()
{
return new Dynamic_library();
}
extern "C" Q_DECL_EXPORT int call_sum(int len, int * data)
{
return Dynamic_library().sum(len,data);
}
Dynamic_library::Dynamic_library()
{
}
int Dynamic_library::sum( int len, int *data )
{
int sum = 0;
for(int i=0; i<len; ++i )
sum += data[i];
return sum;
}
If the target library itself, or at least its specification, is under your control, then you shouldn't be using QLibrary - use the Qt plugin system instead. It doesn't require the call-via-pointer gymnastics otherwise needed.
If you insist on using a dlopen-like mechanism, there is nothing C-specific about QLibrary. The obvious limitation is that the library that you're trying to open must have been compiled with a C++ compiler that's ABI-compatible to the one you use to compile your own code. On Windows that really means using the same MSVC version.
Apart from that, you'll have to look up the mangled version of the symbol. Once you've done that, you can call the symbol using a function/method pointer that matches it. This won't work on constructors/destructors, by design. If you wish to create new instances of objects, you'll need a static factory method provided by the library.
If the library doesn't provide factory methods, you can implement a shim library that links to the target library by a generic name and does provide factory methods. You'll still need to call individual methods by function/method pointers.
Create a temporary folder.
Copy the shim library to the temporary folder.
Copy the target library renamed to the generic name, into the temporary folder.
Save the value of LD_LIBRARY_PATH environment variable.
Prepend the temporary folder to LD_LIBRARY_PATH.
Open/load the library.
Restore the saved value of LD_LIBRARY_PATH.
Of course, you must have the header file for whatever interface the library exposes. It can't be, generally, reconstructed given just a dynamic library file - primarily because the mangled symbols don't have full structural information for the used types. For example, even if you can find a constructor for a given class, you won't know how big is the class instance (its sizeof).
Yes it's possible to do what you're describing on most operating systems, but how you do it is dependent on the system and regardless of the system it's definitely a bit more work on your end to make it happen.
The general steps are:
load the library
for each symbol you're interested in within the library, locate it and store to a variable for later use. (This can be done as-needed, rather than right away.)
For example, in pseudo-code (read: this won't compile!) on a *nix type system, lets assume your shared library has this in it:
// I'm declaring this _extern "C"_ to avoid name mangling, making it easier to
// specify a symbol name for dlsym() later
extern "C" int myFunction() {
return 10;
}
Assume this is in a library called libmyFunction.so. Your main application could, for example:
{
void *handle = dlopen("libmyFunction.so", <flags>);
if (!handle) return; // error: cannot locate the library!
int (*func)() = (int (*)())dlsym(handle, "myFunction");
if (!func) return; // error: cannot locate the symbol!
printf("The function returns: %d\n", func());
}
If you need to do this on Windows, the concept is the same but the function calls are different.

How to replace Lua default error print?

I'm implementing Lua as a script language into a Windows application. Due to the application's structure, printout isn't using streamed io, such as stdout and stderror.
I have managed to override the Lua print to fit into my structure...
lua_register(L,"print", cs_print);
...but how do I override all error an debug printouts without using streams? I need to handle this in a function (similar to print).
The only place where Lua writes to stderr is in the panic function that luaL_newstate installs. If you're embedding Lua into your application, make sure you start Lua from a protected call and no panic will ever occur. See http://www.lua.org/source/5.2/lua.c.html#main for ideas.
After a lot of Google I came up with this solution to fetch compiler and runtime error messages as well to redirect the standar Lua print function.
I use C++ Builder but I think it can be useful for anyone looking for the same answer.
The script is running in a TScriptLua object instance and to map a Lua state against the correct script instance, I use a std::map list to look up the object pointer.
// list for mapping Lua state with object pointers.
static std::map<lua_State*,TScriptLua*> LuaObjMap;
Next is an inline function to get the object pointer from a Lua state pointer.
extern "C" {
// Inline to map Lua state pointer to object pointer
static inline TScriptLua* GetScriptObject(lua_State* L) {
return LuaObjMap.find(L)->second;
}
This function will replace the standard Lua print function. The object pointer (f) has a member function in its base class called Out() that will output a char buffer in the associated window control.
// New Lua print function
static int cs_print (lua_State *L) {
// Map Lua state to object
TScriptLua* f = GetScriptObject(L);
if (f) {
int count = lua_gettop(L);
for (int i=1; i <= count; ++i) {
const char *str = lua_tostring(L, i); // Get string
size_t len = lua_rawlen(L,i); // Get string length
// Output string.
f->Out(str,len);
}
}
return 0;
}
This is my error print routine that will display the compiler/runtime error. As for the Lua print function, the f->Out is used again to print the error message.
// Error print routine
static int cs_error(lua_State *L, char *msg) {
// Map Lua state to object
TScriptLua* f = GetScriptObject(L);
// Get error message
AnsiString m = String(msg) + " " + String(lua_tostring(L, -1));
// "print" error message
f->Out(m.c_str(),m.Length());
// Clenaup Lua stack
lua_pop(L, 1);
return 0;
}
} // <--- End extern C
Here is the actual load and run member. After the new Lua state has been created it is associated with this object in the mapping list. Then it loads the script from a Rich edit control with luaL_loadbuffer(), checks for compiler errors and runs the compiled script with lua_pcall().
void __fastcall TScriptLua::Compile(void) {
// Create new Lua state
lua_State *L = luaL_newstate();
// Store mapping Lua state --> object
LuaObjMap.insert( std::pair<lua_State*,TScriptLua*>(L,this) );
// Override Lua Print
lua_register(L,"print", cs_print);
// Load and compile script
AnsiString script(Frame->Script_RichEdit->Text);
if (luaL_loadbuffer(L,script.c_str(),script.Length(),AnsiString(Name).c_str()) == 0) {
if (lua_pcall(L, 0, 0, 0)) // Run loaded Lua script
cs_error(L, "Runtime error: "); // Print runtime error
} else {
cs_error(L, "Compiler error: "); // Print compiler error
}
// Close Lua state
lua_close(L);
// Remove Lua --> object mapping
LuaObjMap.erase( LuaObjMap.find(L) );
}
This isn't my final solution, but it does the trick so far. I think the best thing is to write stream handling into the TScriptLua object so I can register it directly into Lua.

Using a passed by reference string array to runtime loaded dll function

I'm creating a DLL lib that should be used during runtime (i.e. loaded in Windows 7 with LoadLibrary, closed with FreeLibrary and function handles given by GetProcAddress). This is being done in C++ using Borland C++ Builder. All functions in the DLL should receive some value by reference as parameter (normally std::string).
By now the method I'm using to do this is the following (example) (summarized):
typedef void (*HIS_validity)(string &);
//LoadLibrary
HIS_validity fValidity = (HIS_validity) GetProcAddress(frMain->HIS_DLL.hisLibrary,"checkForValidity");
if (fValidity == NULL) return;
string testeValidade;
fValidity(testeValidade);
const AnsiString testeValidade2(testeValidade.c_str());
if (testeValidade2 != "...")
//etc...
In the DLL:
extern "C" void LIBRARY_API checkForValidity(string &str);
void checkForValidity(string &str)
{
str = "...";
}
Now this code is running fine. The problem is that in some functions I want to pass a whole array of strings by reference. Previously I discovered how to pass a string array by reference here and I though it would be just a matter of ajusting things accordingly:
typedef void (*HIS_patientData)(string (&)[32]);
HIS_patientData fPatientData = (HIS_patientData) GetProcAddress(frMain->HIS_DLL.hisLibrary,"patientDataFields");
string strDado2[32];
fPatientData(strDado2);
frMain->pluginData.patientData.numProntuario = AnsiString(strDado2[cont1++].c_str());
frMain->pluginData.patientData.pacNome = AnsiString(strDado2[cont1++].c_str());
In the DLL:
extern "C" void LIBRARY_API patientDataFields(string (&str)[32]);
void patientDataFields(string (&str)[32])
{
str[0] = "One";
str[1] = "Two";
str[2] = "Three";
//....
}
But here the problem appears. When I compile and run my application, the same problem always come up: if my function in the DLL has only two data attributed to 'str[]', the code goes one after 'fPatientData(strDado2);' but when I read the content of strDado2[0], it has the value of str[1] and strDado2[1] has NULL inside! By the other hand, if I add three or more attributions to 'str[]' in my DLL function, the software always crash when it comes to 'fPatientData(strDado2);' with a pop-up telling "access violation ... in module libstdc++-6.dll".
And I have no ideia what the problem is :T
Thanks for any help,
Momergil
Ok, it seems I found the answer to all such problems... Namely, I'm trying to return a C++ class (std::string) in a "extern "C"" function. It was just a matter of making it return a standart const char* that everything started to run just fine.
Thanks for the contributors,
Momergil

casting to void* to pass objects to pthread in c++

I'm a little confused about how to pass an object to the pthread_create function. I've found a lot of piecemeal information concerning casting to void*, passing arguments to pthread_create, etc., but nothing that ties it all together. I just want to make sure I've tied it all together and am not doing anything stupid. Let's say I have the following thread class:
Edit: fixed mis-matched static_cast.
class ProducerThread {
pthread_t thread;
pthread_attr_t thread_attr;
ProducerThread(const ProducerThread& x);
ProducerThread& operator= (const ProducerThread& x);
virtual void *thread_routine(void *arg) {
ProtectedBuffer<int> *buffer = static_cast<ProtectedBuffer<int> *> arg;
int randomdata;
while(1) {
randomdata = RandomDataGen();
buffer->push_back(randomdata);
}
pthread_exit();
}
public:
ProtectedBuffer<int> buffer;
ProducerThread() {
int err_chk;
pthread_attr_init(&thread_attr);
pthread_attr_setdetachstate(&thread_attr,PTHREAD_CREATE_DETACHED);
err_chk = pthread_create(&thread, &thread_attr, thread_routine, static_cast<void *> arg);
if (err_chk != 0) {
throw ThreadException(err_chk);
}
}
~ProducerThread() {
pthread_cancel(&thread);
pthread_attr_destroy(&thread_attr);
}
}
To clarify, the data in the ProtectedBuffer class can only be accessed with methods like ProtectedBuffer::push_back(int arg), which use mutexes to protect the actual data.
My main question is: am I using static_cast correctly? And my secondary question is do I need that first line in virtual void *thread_routine(void *arg) where I copy the passed void pointer to a pointer to ProtectedBuffer?
Also, if I've done anything else that might cause problems, I'd appreciate hearing it.
There are a number of problems with your code. For starters, I don't
see where the arg you are casting is declared, so I can't say whether
the case is appropriate.
Perhaps more importantly, thread_routine is a member function, so it
can't be converted to a pointer to a function. The function passed to
pthread_create must be extern "C", so it cannot be a member, period;
it must be a free function declare extern "C". If you want to call a
member function, pass a pointer to the object as the last argument, and
dereference it in the extern "C" function:
extern "C" void* startProducerThread( void* arg )
{
return static_cast<ProducerThread*>( arg )->thread_routine();
}
And to start the thread:
int status = pthread_create( &thread, &thread_attr, startProducerThread, this );
Just don't do this in a constructor. The other thread might start
running before the object is fully constructed, with disasterous
effects.
Also, be very sure that the cast in startProducerThread is to
exactly the same type as the pointer passed into pthread_create. If
you cast to a base class in startProducerThread, then be very, very
sure that it is a pointer to that base class that you pass to
pthread_create; use an explicit cast if necessary (to the type in
startProducerThread, not to void*).
Finally, while not relevant to your actual question: if
ProtectedBuffer has an interface like that of std::vector, and
returns references to internal data, there's no way you can make it
thread safe. The protection needs to be external to the class.
If you want to go this route, I believe you want something like this:
Edit: Based on James Kanze's answer, add a separate activate method to launch the thread after construction is finished.
class GenericThread {
protected:
GenericThread () {
//...
}
virtual ~GenericThread () {}
int activate () {
return pthread_create(..., GenericThreadEntry, this);
}
virtual void * thread_routine () = 0;
#if 0
// This code is wrong, because the C routine callback will do so using the
// C ABI, but there is no guarantee that the C++ ABI for static class methods
// is the same as the C ABI.
static void * thread_entry (void *arg) {
GenericThread *t = static_cast<GenericThread *>(arg);
return t->thread_routine();
}
#endif
};
extern "C" void * GenericThreadEntry (void *) {
GenericThread *t = static_cast<GenericThread *>(arg);
return t->thread_routine();
}
Then, ProducerThread would derive from GenericThread.
Edit: Searching for extern "C" in the C++ Standard. revealed no requirement that a function pointer must point to a function with C linkage to be callable by a C library routine. Since pointers are being passed, linkage requirements do not apply, as linkage is used to resolve names. A pointer to a static method is a function pointer, according to C++ 2011 draft (n3242), Sec. 3.9.2p3:
Except for pointers to static members, text referring to pointers does not apply to pointers to members.
Edit: Mea culpa. The C library will invoke the callback function assuming the C application binary interface. A function with C++ linkage may use a different ABI than the C ABI. This is why it is required to use a function with extern "C" linkage when passing to a callback function to a C library. My sincere apologies to James Kanze for doubting him, and my sincere thanks to Loki Astari for setting me straignt.