What is the story regarding having a process on Linux, which dlopen() multiple shared libraries and the executable and/or the shared libraries compiled with different C++ compilers (e.g. provided by customers or 3rd parties).
Am I going right in the following assumptions:
there is only a single namespace for symbols in a linux process. Symbols are found and resolved only by symbol name. The source of the symbol is random in the presence of an unknown executable (customer supplied) or customer supplied shared libraries.
there is no way to make certain, that STL/boost symbols are being resolved from the correct source, as they are always weak and thus might be overwritten.
What are the implications of using multiple copies of (different) libc++ inside the same process (some of them static)?
I don't expect seperate libraries to be able to talk to each other via a C++ interface but only via a C interface. What I would like is, that one can load SharedLibraries from different vendors into a single process and they do not screw up each other.
I know that this has worked in Windows for decades
Your comment completely changes your question:
I don't expect it to be able to talk to each other via a C++ interface but only via a C interface. What I expect is, that one can load SharedLibraries from different vendors into a single process and they do not screw up each other. (This btw is working on Windows since decades)
This element of behaviour is largely system indipendant. The Windows PE format and Linux ELF are similar enough in design that they don't add any additional constraints or capabilities on this topic. So if your technique was going to work in Windows then it should also do so in Linux, just replacing .dll files for .so files.
Linux has more standardisation around calling conventions than Windows, so if anything you should find that Linux make's this simpler.
Origional Answer
Question:
there is only a single namespace for symbols in a Linux process?
That's correct; there's no such thing as namespaces in Linux's loader.
As you may know, C and C++ are very different languages. C++ has namespaces C does not. When libraries are loaded (in both Linux, Unix and also Windows) there is no concept of namespace.
C++ compilers use name mangling to ensure that names isolated by namespaces in your code, do not compiled when placed as symbols in the shared object. C compilers don't do this and don't need to do it because there are no namespaces.
Question:
Symbols are found and resolved only by symbol name. The source of the symbol is random in the presence of an unknown executable (customer supplied) or customer supplied shared libraries.
Let's replace the word "random" for unpredictable. That's also correct. From Wikipedia:
The C++ language does not define a standard decoration scheme, so each compiler uses its own. C++ also has complex language features, such as classes, templates, namespaces, and operator overloading, that alter the meaning of specific symbols based on context or usage. Meta-data about these features can be disambiguated by mangling (decorating) the name of a symbol. Because the name-mangling systems for such features are not standardized across compilers, few linkers can link object code that was produced by different compilers.
Question:
What is the story regarding having a process on LINUX, which dlopen() multiple shared libraries and the executable and/or the shared libraries compiled with different C++ compilers (e.g. provided by customers or 3rd parties).
You can off course dlopen() a shared object, but dlsym() would be tricky to use because of name mangling. You'd have to inspect the shared object manually to determine the precise symbol name.
What are the implications of using multiple copies of (different) libc++ inside the same process (some of them static)?
If you got that far, then I'd be concerned about memory management first of all. libc++ is responsible for implementing new and delete and converting these into memory requests from the OS. If they behave anything like GNU's malloc() and free(), they might manage their own pool of memory. It'd be hard to prectict what would happen if you called delete on an object that was created by a different libc++.
it seems that parts of this randomness can be avoided by loading the shared libraries using the following flags passed to dlopen():
RTLD_LOCAL
RTLD_DEEPBIND
Related
Traditionally, C++ libraries are made of a header file + the implementation compiled in a binary file (.a, .so, .dylib, .dll, ...). The header file is #included in the source code and the binary part is linked to the final executable.
Will modules from C++20 change such layout? If so, will operating systems have to upgrade the way they distribute their core libraries, e.g. the Standard Library in Linux or other core dlls in Windows?
Absolutely. Modules are a very different type of representing library code to users than conventional header/libraries. The main advantage of a module is to have it parsed to the level of the abstract syntax tree (AST) by the compiler. This only happens once per module -- in contrary to every time you include a particular header file. Thus, one possibility for speedup is to convert very frequent header files into modules and save a lot of compiler time in not re-compiling to AST many times, but just once. AST also works perfectly fine for templates ... it is a generic and complete description of the C++ language.
But this is currently also the main "drawback": ASTs are absolutely compiler dependent. There is no stability between vendors, systems, or even compiler versions. So distributing ASTs makes no sense, at least in the current toolchain environment.
Thus, in my understanding, modules are not easily a replacement for common header/libraries. They are a ideal replacement for lightweight (best header-only) and potentially highly templated code to be included many times in typical programs. Many libraries, foremost the standard libraries, are like this. But there are also many libraries of different design (small API+heavy binary backend). For those, I guess, we will have to continue to use include/libraries as before. But, C++ is backward compatible, so mixing modules with includes is no problem at all.
Also, modules can just wrap include files (e.g. with an API). This is probably not a very powerful usage of modules, but may lead to a more uniform programming pattern. In this case, you would still need to link your "so" or "dylib" or whatever library to the resulting code.
Thus, I believe the future at some point will distribute both, some C++ modules, but also conventional headers+libraries. Modules by themselves can be split into many files (only the "exported" symbols are visible to users). Modules will likely follow very different design guidelines than headers. It very likely will not be any advantage to have "one class per module", but rather "a whole bunch of logically connected code per module".
I am working with dynamic linked libraries (.dll) on windows or shared objects (.so) on Linux.
My goal is to write some code, that can - give the absolute path of the library on the disk - return a list of all exported functions (the export table) of that library and ultimately be able to call those functions. This should work on windows (wit dll) as well as on Linux (with so).
I am writing kind of a wrapper that delgates function calls to the respective library. Therefore I receive a path, a function name, and a list of parameters which I then want to forward. the thing is: I want to know whether the given function exists before trying to call it
From here I found a platform-independent way of opening and closing the library as well as getting a pointer to the function with the given name.
So the thing that remains is getting the names of available functions in the first place.
On this topic, I found this question dealing with the same kind of problem only that it asks for a Linux specific solution. In the given answer it is said
There is no libc function to do that. However, you can write one yourself (or copy/paste the code from a tool like readelf).
This clearly indicates that there are tools to do what I am looking for. The only question is, is there one that can work on windows as well as on Linux? If not how would I go about this on my own?
Here is a C# implementation (actually this is the code I want to port to C++) doing what I want (though windows only). To me, this appears as if the library structure is handled manually. If this is the way to go then where can I find the necessary information on the library structure?
So, on unixoids (and both Linux and WinNT have a posix subsytem), the dlopen function can be used to load a dynamic library and get function pointers to known symbols by name of that symbol.
Getting a list of symbols as far as I know was never an aspect that POSIX bothered to specify, so long story short, the functions that can do that for you on Linux are specific to the libc used there (GNU libc, mostly), and on Windows to the libc used there. Portable code means having to different codebases for two different libcs!
If you don't want to depend on your libc, you'd have to have a binary object parser (for ELF shared libraries on Linux, PE on Windows) to read out symbol names from the files. There's actually plenty of those – obviously, WINE has one for PE that is portable (especially works on Linux as well), and every linker (including glibc's runtime linker) under Linux can parse ELF files.
Personally, radare2 is a fine reverse-engineering framework with plenty of language bindings that is actually meant to analyze binary files, and give you exported symbols (as well as being capable of extracting non-exported functions, constructing call-graphs etc). It does have debugger, i.e. jumping-into-functions capabilities, too.
So, knowing now that
I am writing kind of a wrapper that delgates function calls to the respective library. Therefore I receive a path, a function name, and a list of parameters which I then want to forward. the thing is: I want to know whether the given function exists before trying to call it
things become way easier: you don't actually need to get the list of exports. It's easier and just as fast to simply try.
So, on any POSIX system (and that includes both Windows' POSIX subsystem and Linux), dlopen will open the library and load the symbol table, and dlsym will look up a symbol from that table. If that symbol is not in the table, it simply returns NULL. So, you already have all the tables you think you need; just not explicitly, but queryable.
I started to use some custom memory allocators such rpmalloc and ltmalloc into my project, but I have some concern about the integration, my project has various internal modules built as shared libraries or static libraries (depending how I configure them in my build system) and should build/run for the Windows/Linux/FreeBSD/Mac OS X, and architectures such x86 and ARM, and I don't know if I should make the calls of my memory allocator integrations inside of the header file or should remain inside of the cpp file.
If the memory allocator calls stay in the header files, every module should link the static library of the memory allocator, if it is kept in the .cpp file the calls are contained in the library who contains them, and only that module should link the custom memory allocator, but that module should contain an interface to every module can allocate them (avoiding memory allocation inconsistency)
I've read here if the memory is allocated normally (like malloc/free/syscalls does) every shared library has his own heap, but if uses mmap allocates memory which doesn't belong to the program's heap.
My question is does it introduce any hazard in my shared/static libraries if they are kept into one library (but every other library should link it in order to access their memory allocation interfaces)? or should everything inline in the header files, and every library should link the memory allocator library?.
How memory allocation is done heavily depends on the operating system. You need to understand how shared libraries work in those operating systems, how C language relates to those operating systems and to the concept of shared libraries.
C, C++ and modular programming
First of all, I want to mention that C language is not a modular language e.g. it has no support for modules or modular programming. For languages like C and C++ implementation of modular programming is left to the underlying operating system. Shared libraries is an example of mechanism that is used to implement modular programming with C and C++, therefore I will refer to them as modules.
Module = shared library and executable
Linux and Unix-like systems
Initially everything on Unix systems was statically linked. Shared libraries came later. And as Unix was a starting point for the C langauge, those systems try to provide shared library programming interface that is close to what programming in C feels like.
The idea is that C code written originally without shared libraries in mind should be build and should work without changes made to the source code. As the result, provided environment usually has single process-wide symbol namespace shared by all loaded modules e.g. there can only be a single function with a name foo in the whole process, except for static functions (and some functions that are hidden in moduels using OS-specific mechanisms). Basically it is the same as with static linking where you are not allowed to have duplicate symbols.
What this means for your case is that there is always a single function named malloc in use in the whole process and every module is using it e.g. all modules share the same memory allocator.
Now if process happens to have multiple malloc functions, only a single one is picked and will be used by all modules. Mechanism here is very simple - as shared libraries do not know location of every referenced function, they will usually call them though some table (GOT, PLT) that will be filled with required addresses lazily on the first call or at load time. The same rule is applied to the module that provides original function - even internally this function will be called though the same table, making it possible to override that function even in the original module that provides it (which is the source of many ineffeciencies related to usage of shared libraries on Linux, search for -fno-semantic-interposition, -fno-plt to overcome this).
The general rule here is that the first module to introduce symbol will be the one providing it. Therefore original process executable has the highest priority here and if it defines malloc function, that malloc function will be used everywhere in the process. The same applies to functions calloc, realloc, free and others. Using this trick and tricks like LD_PRELOAD allow you to override "default memory allocator" of your application. This is not guaranteed to work thought as there are some corner cases. You should consult documentation for your library before doing this.
I want to specifically note that this means there is a single heap in the process shared by all modules and there is a good reason for that. Unix-like systems usually provide two ways of allocating memory in a process:
brk, sbrk syscalls
mmap syscall
The first one provides you an access to a single per-process memory region usually allocated directly after the executable image. Because of the fact that there is only one such region, this way of memory allocation can only be used by a single allocator in a process (and it is usually already used by your C library).
This is important to understand before you throw any custom memory allocator into your process - it either should not use brk, sbrk, or should override existing allocator of your C library.
The second one can be used to request chunk of memory directly from the underlying kernel. As kernel know the structure of your process virtual memory, it is able to allocate pages of memory without interfering with any user-space allocator. This is also the only way to have multiple fully independent memory allocators (heaps) in the process.
Windows
Windows does not rely on C runtime the same way Unix-like systems do. Instead it provides its own runtime - Windows API.
There are two ways of allocating memory with Windows API:
Using functions like VirtualAlloc, MapViewOfFile.
And heap allocation functions - HeapCreate, HeapAlloc.
The first one is an equivalent to mmap, while the second one is a more advanced version of malloc which is based internally (as I believe) on VirtualAlloc.
Now because Windows does not have the same relation to C language as Unix-likes have, it does not provide you with malloc and free functions. Instead, those are provided by C runtime library which is implemented on top of Windows API.
Another thing about Windows - it does not have a concept of single per process symbol namespace e.g. you cannot override function here the same way you do on Unix-like systems. This allows you to have multiple C runtimes co-existing in the same process, and every of those runtimes can provide its independent implementation of malloc, free etc, each operating on a separate heap.
Therefore on Windows all libraries will share a single process Windows API-specific heap (can be obtained through GetProcessHeap), at the same time they will share heap of one of C runtimes in the process.
So how do you integrate memory allocator into your program?
It depends. You need understand what you are trying to achieve.
Do you need to replace memory allocator used by everyone in your process e.g. the default allocator? This is only possible on Unix-like system.
The only portable solition here is to use your specific allocator interface explicitly. It doesn't really matter how you do this, you just need to make sure the same heap is shared by all libraries on Windows.
The general rule here is that either everything should be statically linked or everything should be dynamically linked. Having some sort of mix between the two might be really complicated and requires you to keep the whole architecture in your head to avoid mixing heaps or other data structures in your program (which is not a big problem if you don't have many modules). If you need to mix static and dynamic linking, you should build you allocator library as a shared library to make it easier having single implementation of it in a process.
Another difference between Unix-alikes and Windows is that Windows does not have a concept of "statically linked executable". On Windows every executable has dependencies on Windows-specific dynamic libraries like ntdll.dll. While with ELF executables have separate types for "statically linked" and "dynamically linked" executables.
This is mostly due to single per-process symbol namespace which makes it dangerous to mix shared and static linking on Unix-alikes, but allows Windows to mix static and dynamic linking just fine (almost, not really).
If you use one of your libraries, you should make sure you link it dynamically with dynamically linked executables. Imagine if you link your allocator statically into your shared library, but another library in your process uses the same library too - you might be using another allocator by accident, not the one you were expecting.
I am working on code that creates 'models'. A model is created from an XML file and part of its representation is, generated on the fly, C code. This C code is compiled, on the fly, into a shared library that is dynamically loaded (using POCO shared lib class). The shared library mainly contains small functions, and part of a models creation is to populate function pointers to these functions. All this works fine. However, creating several models, at the same time, causes problems.
I believe it has to do with how the dynamic loading works on Linux, and the fact that each shared library contain functions with identical names. Does PIC cause this? The problems manifest itself in no sense data being retrieved from shared libraries functions.
So question is, how to load multiple (thousands) shared libraries, containing identical function names, on linux?
The above works fine on windows, where it seems that dynamically loaded libraries data/functions are kept perfectly isolated from each other.
Poco SharedLibrary open function accepts linker flags. Default is Poco::SharedLibrary::SHLIB_GLOBAL, corresponding to dlopens RTLD_GLOBAL and Poco::SharedLibrary::SHLIB_LOCAL, corresponding to RTLD_LOCAL. See http://linux.die.net/man/3/dlopen for more info.
Passing the Poco::SharedLibrary::SHLIB_LOCAL flag fixed the problem.
First, you can dlopen many hundreds of thousands of shared objects. My manydl.c demonstrate that.
Then, you can also generate C code, compile it, and dlopen the shared object, all this from the same process. My (obsolete in 2017) MELT plugin (for GCC, MELT provides a high level language to extend GCC) does that (and so does my manydl.c example).
However, I don't think you should keep identical (defined) function names in them. I suggest avoiding that. You could
generate unique names (since the C code is generated, this is the best, most portable, and simplest solution)
compile with some -D flags to #define these names to unique names, so the source code could apparently contain duplicate names; that is if your generated code defines the foo function pass a -Dfoo=foo_123 (with foo_123 being globally unique) to the gcc command compiling it. (Of course you then dlsym for "foo_123").
add visibility("hidden") function attributes in your generated code. You could also pass -fvisibility=hidden option to gcc.
have only static functions (then, the name don't matter much so can be duplicate), and have a constructor function which somehow binds the functions (e.g. store their pointer somewhere, e.g. in some global table).
you might consider passing RTLD_LOCAL to dlopen(3). I'm not sure it is a good idea (and POCO probably don't know how to do that).
PS. I don't think it is related to position independent code (which is preferable, but not absolutely required, in shared objects; without -fPIC the dlopen would have to do a lot of inefficient relocations). It is related to Linux shared object linking & loading. Read Levine's Linkers & Loaders book for details.
See also this question, and read Drepper's paper on How to Write Shared Libraries.
I have two "plugins" (think of them just as two different applications in a software package for discussion sake) that dynamically link to two separately built versions of my library. My code is written in C++ and uses consistent namespaces throughout. Sometimes I have to build two different
versions for each application. This appears to cause some problems when both applications (plugins)
in the package are loaded simultaneously. First I need help understanding why this error occurs.
As an example, I have two separate but identically named libraries, say mylib.so (or DLL) and each application is linking to (unique) one of these. If the underlying code in mylib.so is the same (i.e., namespaces, names of functions, etc. with slightly different implementations of course) should this cause problems? Isn't the fact that the two copies of the libraries are located in unique locations sufficient to avoid any issues that may arise due to ambiguity or other linking errors? I think clearly not.. but I'd like to hear from an expert on this.
Assuming the description above is what's causing the problem, will merely changing the names of the libraries to include say some version information, e.g., mylib_v1.so and mylib_v2.so provide a safeguard against ambiguity errors (underlying function/namespaces names being still identical)? I still think not.. but I am not sure this time. Assuming I am right, would changing the namespaces using some macro in my code to include version info in namespaces (e.g., namespace mystuff {} changed to namespace mystuff_v1) do the trick at least? Appreciate your insights.
NOTE: Surprisingly the ambiguity occurs only on Windows! Linux is able to handle the situation in second paragraph without any problems.
If the applications are uniquely only using one, and PATHs and LD_LIBRARY_PATHs are set
so that they do not meet there is no clash. You can see that with the hundreds of similar
msvcrt.dll files once distributed with the applications, until Microsoft sorted that out.
But still, your (slightly different) code may create or refer to global resources and here
can be the collision. Sure the Windows variant doesn't use something globally named, and
introduces different data structures here? (File to store standard settings, shared memory,
...). As this global stuff is mostly pretty system-dependent - maybe you do something on Windows you don't do on Linux ...