I have a huge map that is needed in multiple shared objects ..
//mymap.h
extern const std::unordered_map<int, std::string> gMAP;
std::string getname(int i) { return gMAP[i]; }
//mymap.cpp
extern const std::unordered_map<int, std::string> gMAP = {
{1, "one" },
{2, "two" },
{3, "three" },
...
}
And then I have library1 (lib1.so):
//CMakeLists.txt lib1.so
add_library(lib1 SHARED lib1.cpp mymap.cpp)
//lib1.cpp
#include "mymap.h"
void lib1_dosomething(int i) { std::cout << getname(i) << std::endl; }
And then I have library2 (lib2.so):
//CMakeLists.txt lib2.so
add_library(lib2 SHARED lib2.cpp mymap.cpp)
//lib2.cpp
#include "mymap.h"
void lib1_dosomething(int i) { std::cout << getname(i) << std::endl; }
This is simply pseudo code of our complex and much bigger system. Note that not all the libraries in our system need the map. Only very few (which is why I went for compiling in the project itself).
My understanding of making it this way is as follows:
Each library that compiles "mymap.cpp" has a 'new' instance of the map
There shall be no conflicts when all the libraries are loaded into memory when the system runs. And each library creates the global map when it is loaded and destroys it when it is unloaded from memory.
So my first question is: Is this understanding correct ?
But, I see a problem during the shutdown of our system. I noticed that each library creates an instance as expected, and destroys its instance during unloading. But one library seems to call the destructor of the std::unordered_map twice. And as a result we see a deallocation error (double free or corruption (!prev)), which i cannot explain why and what is causing this. Any ideas ?
I am thinking of compiling the map in a shared library of itself, so that we only have absolutely one instance throughout the system, but i want to understand why I see the "double free or corruption (!prev)" error.
Related
I am converting a large Windows C++ application from a large set of source files to
a smaller core application linked to several static libraries (to which many
of the original source files are moved).
Each library requires access to a 'registration method' in the core application.
Each library should call that method during global initialization, but that is
not happening. That is my problem.
The code works fine in the original form where libraries are not used.
I guess I am omitting a necessary link option for the libraries, but
I don't know which.
I have created a minimal, workable example. I developed this
on Windows 10 using:
CMake 3.14.5
MSVC 2019
Here's CMakeLists.txt:
cmake_minimum_required(VERSION 2.8.9)
project (CMakeLinkTest)
add_library(myLibrary STATIC MyStar.cpp)
add_executable(CMakeLinkTest StarFactory.cpp main.cpp)
target_link_libraries(CMakeLinkTest myLibrary)
The application contains main.cpp:
#include <iostream>
int main(int argc, char *argv[]){
std::cout << "Hello World!" << std::endl;
return 0;
}
and a singleton class called StarFactory.
StarFactory.h:
#include<string>
class StarFactory
{
public:
static StarFactory* instance();
~StarFactory() {};
std::string registerStarType(std::string a_type);
private:
StarFactory() {};
static StarFactory* mp_instance; // Singleton instance
};
StarFactory.cpp:
#include <iostream>
#include "StarFactory.h"
StarFactory* StarFactory::mp_instance = 0;
StarFactory* StarFactory::instance()
{
if ( mp_instance==0 )
mp_instance = new StarFactory;
return mp_instance;
}
std::string StarFactory::registerStarType(std::string a_type)
{
std::cout << "registerStarType: " << a_type << std::endl;
return a_type;
}
Finally, a static library contains class MyStar which registers itself with
the singleton at global initialisation.
MyStar.cpp:
#include<string>
#include "StarFactory.h"
class MyStar
{
public:
MyStar() {
StarFactory* s = StarFactory::instance();
//s->registerStarType("MyStar");
};
};
MyStar myStar;
std::string starName = StarFactory::instance()->registerStarType("MyStar");
Now for what happens. If I link MyStar.cpp directly into the application I see:
>CMakeLinkTest.exe
registerStarType: MyStar
Hello World!
If link MyStar.cpp into MyLibrary.lib and link that to the application I see:
>CMakeLinkTest.exe
Hello World!
So, the library's call (last line of MyStar.cpp) to the application's singleton is not working.
Can anyone explain this please?
As stated by engf-010, if a symbol defined in your static library is not used, the linker won't put it in the final binary.
One way to solve the problem using CMake would be to use an OBJECT library instead of a STATIC library.
The default behavior for linker is to not include static library that is not referenced.
You can either:
force linker to include the library anyway - you can use cmake's add_link_options or equivalent
not use a static library - just link the object like as in your first example
reference the code in static library
use a shared object (dynamic library) instead
I have some code that is compiled as a shared library and used with a universal driver, which can be used with other shared libraries that are specific to a particular application.
My question pertains to obtaining some sort of indicator of the name of the binary containing a code that lives in that shared library.
For example, let's say I have 3 files, the first is driver.cpp, the universal driver:
#include "interface.h"
#include <stdio.h>
int main(int argc, char *argv[]) {
//perform a function from the shared library
std::cout << foobar() << std::endl;
}
The second is sharedlibrary.cpp, the specific implementation for one case of many:
#include "interface.h"
char* foobar() {
return x;
}
Where x is some indicator that this function is defined in sharedlibrary.cpp, or that this function is linked from sharedlibrary.so, or the current stack frame is using the specific binary rather than just being included in driver.cpp.
The last file is interface.h, which provides the interface to the library via extern "C"
extern "C" {
char foobar();
}
I would like to reiterate, for clarity, that I am looking for some indication that this function is being linked from sharedlibrary.so. Many solutions looking for runtime filenames give the executable name using either argv[0] or readlink(), but I have no control over the actual naming of driver.cpp or its executable name. Rather, I can distribute sharedlibrary.so, and would like to be able to use its name from within itself, if possible.
If it helps, I know that a microsoft-specific solution could be to use AfxGetApp()->m_pszAppName to obtain the DLL name. However, I am looking for a linux solution that does not necessarily need to be portable.
EDIT: I do not know or control the names of driver.cpp, sharedlibrary.cpp, or sharedlibrary.h at compile time. I wish to discover the name of sharedlibrary.cpp at run time.
The updated sharedlibrary.cpp with x replaced with the solution looks like this
#include "interface.h"
#include <dlfcn.h>
void func() {
//Some function that is defined in sharedlibrary.cpp
}
char* foobar() {
Dl_info DlInfo;
if(!dladdr((void*)func, &DlInfo)) {
return "default_name";
}
return DlInfo.dli_fname;
}
Obtaining filename at runtime for a shared library c++
My question pertains to obtaining some sort of indicator of the name of the binary containing a code that lives in that shared library.
You can use int dladdr(void *addr, Dl_info *info. It fills a following structure for you:
typedef struct {
const char *dli_fname; /* Pathname of shared object that contains address */
void *dli_fbase;
const char *dli_sname;
void *dli_saddr;
} Dl_info;
You can pass the address of a function exported by the shared library as the argument addr. Or within such function, you could use the instruction pointer value of the current stack frame - if you know how to obtain it.
I believe you must link with the libdl library.
You can use the buildsystem to generate the dynamic library name for linking and preprocess that inside of a header with a function that return a defined macro, in cmake you can see how to do that here.
Then you use the configured-file to return the defined value in a function that's exported from within the dll.
#include "library_name_macro.h"
auto __dllexport libraryName() -> std::string { return LIBRARY_NAME_MACRO; }
I hope, I have understood your question correctly. I hope my answer helps. You know the shared library name, you link that shared library to your program, Later in run time you want to figure out whether a particular function is present in library or not and this logic should be part of shared library itself.
Let's take an example that you have shared library called librandom.so, You have linked this library to your application. You can implement the following function in a librandom.so library, You can pass function name which you want to check whether it is present or not. I have not tested this code, there may be errors. The idea I am proposing is library loads itself again to check whether the function is present when this function is called. May not be ideal method but should serve your purpose.
int isFuncPresent(char funcName[])
{
int isFuncFound = 1;
void *lib_handle;
int x;
char *error;
lib_handle = dlopen("librandom.so", RTLD_LAZY);
if (!lib_handle)
{
fprintf(stderr, "%s\n", dlerror());
isFuncFound = 0;
}
fn = dlsym(lib_handle, funcName);
if ((error = dlerror()) != NULL)
{
fprintf(stderr, "%s\n", error);
isFuncFound = 0;
}
dlclose(lib_handle);
return isFuncFound;
}
In a main program, I dlopen and dlclose (LoadLibrary and FreeLibrary respectively) a shared library. The shared library contains a static variable that is instantiated upon dlopen, and destroyed upon dlclose. This behavior is consistent on MSVC 2008 and 2013, GCC 3.4.6, and Sunstudio 12.1.
With GCC 4.9.1 and GCC 5.2.1 however, the destructor was no longer called on dlclose. Instead, it was called before program exit.
The particularity of the static variable's class, is that in the constructor, there is a call to a templated function get (of global scope) that returns a local static variable.
I was able to reproduce this behavior with the following one cpp file linked into a shared library:
#include <iostream>
template <typename T> // In my actual code, i is of type T, however, this has no effect
int get()
{
static int i = 0;
return i;
}
class Dictionary {
public:
Dictionary()
{
std::cout << "Calling Constructor" << std::endl;
get<int>();
}
~Dictionary(){
std::cout << "Calling Destructor" << std::endl;
}
private:
Dictionary(const Dictionary&);
Dictionary& operator=(const Dictionary&);
};
static Dictionary d;
I investigated the tweaks that can be made in order to have the destructor called on dlclose, and concluded the following:
If the function get was not templated
else if the variable i in the function get was not static
else if the function get is made static
The main program's code is the following:
#include <dlfcn.h>
#include <cassert>
#include <string>
#include <iostream>
void* LoadLib(std::string name)
{
void* libInstance;
name = "lib" + name + ".so";
libInstance = dlopen(name.c_str(), RTLD_NOW);
if ( ! libInstance ) std::cout << "Loading of dictionary library failed. Reason: " << dlerror() << std::endl;
return libInstance;
}
bool UnloadLib(void* libInstance)
{
int ret = dlclose(libInstance);
if (ret == -1)
{
std::cout << "Unloading of dictionary library failed. Reason: " << dlerror() << std::endl;
return false;
}
return true;
}
int main()
{
void* instance = LoadLib("dll");
assert(instance != 0);
assert(UnloadLib(instance));
std::cout << "DLL unloaded" << std::endl;
}
I built the binaries with the following commands:
g++ -m64 -g -std=c++11 -shared -fPIC dll.cpp -o libdll.so
g++ -m64 -g -std=c++11 -ldl main.cpp -o main.out
The output I get when the destructor is called before program exit is the following:
Calling Constructor
DLL unloaded
Calling Destructor
The output I get when the destructor is called on dlclose is the following:
Calling Constructor
Calling Destructor
DLL unloaded
Questions:
If the change of behavior between the versions of GCC is not a bug, can you please explain why is the destructor not called on dlclose?
Can you please explain for each tweak: Why is the destructor called on dlclose in this case?
There is no guarantee that unloading (destructors are invoked) happens on dlclose. On musl (as opposed to glibc), constructors only run the first time a library is run, and destructors only run on exit. For portable code, dlclose cannot be assumed to unload the symbols immediately.
The unload behavior depends on glibc's symbol binding when doing dynamic linking, and is independent of GCC.
The static variable get::i has a STB_GNU_UNIQUE binding. For static variables in inline functions, the uniqueness of the object is assured by the ELF linker. However, for dynamic loading, the dynamic linker assures uniqueness by marking the symbol STB_GNU_UNIQUE. Hence, another attempt to dlopen the same shared library by some other code will lookup the symbol and find that it is unique and return the existent one from the unique symbols table. A symbol with unique binding cannot be unloaded.
Unique binding can be disabled with -fno-gnu-unique if not needed.
References
Bug that I raised to GCC
STB_GNU_UNIQUE
I'm trying to implement a design similar to the one described here:
http://www.linuxjournal.com/article/3687
I've got a core projects which compiles into an executable binary. I also want to have shared libraries loaded at runtime using dlopen(). Now I'm facing the problem that there are some classes which should be known in the core project and also in all of the loaded libraries. I thought I'd put them together in another shared library which gets linked to all of them at compilation. Everything seems to compile fine but during loading the shared library ConnectionModule.so dlopen returns no handle and dlerror says 'undefined symbol: _ZTV17PipelineProcessor' where PipelineProcessor is a class defined and implemented in the shared library which should be compiled in every addition shared library.
Does anyone have an idea of what is wrong with my implementation or is this design just doomed to fail?
some code here:
loading of library at runtime
#include "../SharedHeaders/SharedInEveryLibrary.h"
// ...
map<string, maker_t*, less<string> > module_library;
// ...
void *hndl = dlopen(library_file_path, RTLD_NOW | RTLD_GLOBAL);
if (hndl == nullptr) {
Logger::error << "Could not load library " << library_file_name << ": " << dlerror() << endl;
exit(-1);
}
ConditionModule.h (should be loaded at runtime)
#pragma once
#include "../SharedHeaders/SharedInEveryLibrary.h"
class ConditionModule : public B {
public:
A* processRequest(X* data) override; // implemented at ConditionModule.cpp
};
extern "C" {
A *maker() {
return new ConditionModule;
}
class proxy {
public:
proxy() {
module_library["condition_module"] = maker;
}
};
proxy p;
}
... and of course aome of the content of SharedInEveryLibrary.h
class Z; // gets implemented later, I guess it's not important
struct X {
int something;
// ...
};
class A {
public:
virtual void setSomeData(X* v_some_data);
virtual A* process(Z* data) = 0;
protected:
X* some_data;
};
class B : public A {
public:
A* process(Z* data) override;
virtual A* processRequest(X* data) = 0;
};
typedef A* maker_t();
extern map<string, maker_t*, less<string> > module_library;
EDIT -----
Forgot to mention that I currently develop only with Linux as compilation target.
There is nothing wrong with this design. But when you dlopen() a shared library, all it's symbols need to be resolved. Your error means that you do not have the path to .so your library depends on in LD_LIBRARY_PATH. To check what library it needs, use
ldd <your library.so>
Than add the directory where this library resides to LD_LIBRARY_PATH.
When building a dynamic library (in C++), we can select Multi-threaded Debug (/MTd) or Multi-threaded Debug DLL (/MDd) run time library for Windows. If we select Multi-threaded Debug, then the created dynamic library will be in charge of memory allocation for all the variables that are used in the library. Therefore, the following example will show a case where /MDd setting will work while /MTd setting will fail:
my_dll.h
class MY_EXPORT DllClass
{
public:
std::vector<int> abcWorkable;
void create_new_input_workable();
};
my_dll.cpp
void DllClass::create_new_input_workable()
{
abcWorkable.push_back(3);
abcWorkable.push_back(4);
}
main.cpp
int main(void)
{
DllClass mine;
//mine.abcWorkable.reserve(20);
mine.create_new_input_workable();
return 0;
}
There are two work-around solutions to this problem: one is using static library rather than the dynamic one and the other is to allocate memory only in the dynamic library or the executable, for example if we change main.cpp:
int main(void)
{
DllClass mine;
mine.abcWorkable.reserve(20);
mine.create_new_input_workable();
return 0;
}
It will work as this time the variable std::vector<int> abcWorkable is allocated memory in the executable program. However, if the variable insider the class (in the dll) is difficult to allocate memory in the executable, then this solution may fail. I also take another example:
my_dll.h
class MY_EXPORT DllClass
{
public:
std::list<std::vector<int> > myContainer;
void create_new_input();
}
my_dll.cpp
void DllClass::create_new_input()
{
std::vector<int> abc;
abc.push_back(2);
abc.push_back(3);
myContainer.push_back(abc);
}
main.cpp
int main()
{
DllClass mine;
mine.create_new_input();
std::list<std::vector<int> >::iterator it = mine.myContainer.begin();
std::list<std::vector<int> >::iterator itEnd = mine.myContainer.end();
while(it != itEnd)
{
for(int i=0; i<(*it).size(); i++)
std::cout<<(*it)[i]<<std::endl;
it++;
}
return 0;
}
It is impossible to allocate memory before hand for the variable std::list<std::vector<int> > myContainer, then my question is how I can deal with this situation? (Except using static library) Thanks.
When you build with /MT then every module will have its own copy of the CRT. With its own global variables like errno and its own memory allocator that uses its own heap. The final program will run with multiple CRT copies.
This is very troublesome, particularly with your code. Because you return an std::string by value from the DLL. Which was allocated on the heap that the CRT copy that the DLL uses. This object needs to be released again by the caller. But it uses a different heap and cannot possibly release the object. Kaboom.
Using /MD is a hard requirement here. Which ensures that every module uses the same CRT copy and only a single implementation is present in the final program. As long as they were built with the same CRT version anyway.