When developing a C/C++ (=2?) plugin based framework with shared objects/dynamic libraries that need to support live swapping what examples would be helpful to look at for implementation details?
Thanks.
Note: live swapping is the key point here, no need to restart the system is a requirement
If you are on POSIX, dlopen(), dlsym() and dlclose() are all you need.
See man dlsym for details and examples.
There is a good article about loading dynamic libraries, and plugin infrastructure is an example.
EDIT OP added Windows as requirement so this approach won't help since Windows isn't POSIX-compliant. However there are similar functions in WinAPI - see here.
You might want to try Boost.Extension but beware : despite its name, it is not one of boost libraries.
Here is a link to its documentation.
For C++ plugins you can check this article which detail how to achieve it with the previously mentionned posix calls.
Quoting the article :
Given that we can use these functions to access functions in a C library, how do we use them to access classes in a C++ library? There are several problems to overcome. One is that we must be able to locate the symbols we need in the library. This is trickier than it might seem because of the difference between the way symbols are stored in C and C++ files.
If you want cross-platform library loading without having to develop for each platform's API individually, libltdl may help.
Libtool provides a small library, called libltdl, that aims at hiding the various difficulties of dlopening libraries from programmers. It consists of a few headers and small C source files that can be distributed with applications that need dlopening functionality. On some platforms, whose dynamic linkers are too limited for a simple implementation of libltdl services, it requires GNU DLD, or it will only emulate dynamic linking with libtool's dlpreopening mechanism.
libltdl supports currently the following dynamic linking mechanisms:
dlopen (Solaris, Linux and various BSD flavors)
shl_load (HP-UX)
LoadLibrary (Win16 and Win32)
load_add_on (BeOS)
NSAddImage or NSLinkModule (Darwin and Mac OS X)
GNU DLD (emulates dynamic linking for static libraries)
libtool's dlpreopen (see see Dlpreopening)
Boost.Extension seems to only support Windows PE dlls, UNIX ELF shared objects, and Mac OS X Mach-O bundles. Well, that may be sufficient for you...
Boost.Extension seems nice (never used it but will try soon). Another alternative would be the POCO SharedLibrary class.
Related
I wrote a program, that uses a shared library installed on my system. This library is seldom installed on other systems. How do I compile my program so that the library doesn't need to be installed on other systems? I have the source code for the library available. What's the best way?
The other systems of course have the same architecture and OS.
Compile it as a static library and link that into the executable.
Though the OP had solved his problem by answering a different question, there are (at least) two ways to wedge a shared library into your binary in case
there is no source code available
there is no compiler (or build-chain) available
static link does not work or it's not obvious how do it
to preserve memory layout - static link will change it and may "wake-up" hidden bugs
for "permanent link" LD_PRELOAD library into executable
The first is statifier (open source but limited to x86 and x86_64 and only object code)
The second that I know of is magic ermine (by the same developer). It is closed source, but the developer is friendly to opensource projects and ermine has the advantage of supporting more platforms as well as the ability to include all necessary data files within its virtual file system.
http://statifier.sourceforge.net/ and http://www.magicermine.com/
I have a question related to embedding one library in another.
I have a code that is pure C and my users rely on that, they don't want to depend on C++ libraries. However, the need arose to embed a 3rd party library (ICU) into mine. None of the ICU functions would be exported, they would only be used internally in my library. Unfortunately ICU is a C++ library, though it does have a C wrapper. ICU does not use exceptions, but it does use RTTI (abstract base classes).
The question is how could I create my static library so that
ICU is embedded in my library (all references to ICU functions are resolved within my library)
all references to libstdc++ are also resolved and the necessary code is embedded into my library
if a user does not even have libstdc++ installed on their system things work just fine
if a user does happen to use my library within a C++ project then there are no conflicts with whatever libstdc++ (presumably the system libstdc++) he uses.
Is this possible at all? The targeted platforms are pretty much everything: windows (there my library is dynamic), and all sort of unix versions (linux, solaris, aix, hpux - here my library needs to be static).
gcc-4.5 and later does have --static-libstdc++, but as far as I understand it is only for creating shared libs or executables, and not static libs.
Thanks for any help!
The solution to this problem is pretty simple, but may not fall inside the parameters you have set.
The simple rules are:
You can dynamically link a C++ library to a C caller, but you have to wrap the library inside an extern C layer. You design the extern C API and implement it using C++ internals, which are forever hidden from view. [You can also use COM or .NET to do this on Windows.]
You cannot statically link a C++ library to a C caller. The libraries are different and the calling sequences/linker symbols are different. [You often can't even statically link between different versions of the same compiler, and almost never between different compilers.]
In other words, the solution is simple: use dynamic linking. If that's not the right answer, then I don't think there is one.
Just to make things interesting, you can even implement your own plug-in architecture. That's just another name for dynamic linking, but you get to choose the API.
Just to be clear, the only viable portable option I can see is that you link ICU inside its own dynamic library (DLL or SO). Its symbols, C++ libs, RTTI and exceptions all stay inside that. Your static lib links to the ICU dynamic lib by extern C. That's exactly how much of Windows is built: C++ inside DLL, extern C.
You can debug across the boundary, but you cannot export type information. If you need to do that, you will have to use a different API, such as .NET or COM.
I don't know if this will work, but let me at least suggest that you try it!
The excellent LLVM project (origin of clang compiler) has many front-ends and back-ends for different languages, such as C++ and C. And according to this S.O. question it should be possible for LLVM to compile C++ into C which in turn can be compiled as normal.
I imagine this route is a bumpy one, but if it works, it might solve your problem without the need to link dynamically. It all depends on if ICU will compile with LLVM C++.
If you do decide to give it a go, please let us know how you fare!
I am looking to create a C++ library that can be used by both Linux and Windows clients. The OS specific functionality will be hooked up by the client by implementing the interfaces provided by the library.
Is this possible to achieve? Do I need to recompile the C++ project again in linux.
P.S: I am using CodeBlocks IDE
The short answer is no, you still need to compile your library for each targetted platform -- however, assuming your code is written such that it is cross-platform, you can set up your build to target both Windows and Linux environments with little fuss. I do this now using CMake to generate both Visual Studio projects for Windows environments and Makefiles for Linux environments.
I'm pretty confident that Linux will not accept a .dll :) And yes, you will need to recompile. Unless you run windows as a virtual machine under linux which sort of preempts the question.
It certainly cannot be the same binary file: shared objects ELF format on Linux, DLL "PE" format on Windows. And dynamic loading has different semantics on both systems. See Levine's linker and loader book for details.
You could, if done carefully, have the same source code giving the two different files (the DLL on Windows, the dynamic shared object on Linux).
But you probably would need some conditional compilation tricks like #ifdef WINDOWS etc...
You might use libraries providing you a common abstraction for such things. For instance, both GTK/Glib and Qt have some mechanism giving a common abstraction of dynamically linked (or dynamically loaded - ie dlopen-ed) libraries.
You probably want to read the Program Library Howto (at least for Linux).
I was just wondering what my options were for cross-platform implementations for the dynamic loading of plugins using shared libraries. So far the only one that I have found is:
http://library.gnome.org/devel/glib/stable/glib-Dynamic-Loading-of-Modules.html
And I was just wondering if I had other options? Essentially, I want to be able to put plugins in shared object files, and load them at runtime and I wanted to do it in a cross-platform C++ way.
Edit: I found this Dr Dobbs Post from 2007; surely somebody has come up with something more since then.
You could look into Boost Extension, though it has not yet been accepted into Boost.
The Boost.Extension library has been
developed to ease the development of
plugins and similar extensions to
software using shared libraries.
Classes, functions and data can be
made available from shared libraries
and loaded by the application.
Qt has a nice plugin system. You should take a look at the second part of that page.
If you want something simple and lightweight try: https://pocoproject.org/docs/package-Foundation.SharedLibrary.html
Using the SharedLibrary class it takes three lines to call a function in a C shared library:
Poco::SharedLibrary lib("libfoo.so");
int (* foo)(int) = reinterpret_cast<int (*)(int)>(lib.getSymbol("foo"));
printf("answer %d\n", foo(5));
I recently had to program C++ under Windows for an University project, and I'm pretty confused about static and dynamic libraries system, what the compiler needs, what the linker needs, how to build a library ... is there any good document about this out there? I'm pretty confused about the *nix library system as well (so, dylibs, the ar tool, how to compile them ...), can you point a review document about the current library techniques on the various architectures?
Note: due to my poor knowledge this message could contain wrong concepts, feel free to edit it.
Thank you
Feel free to add more reference, I will add them to the summary.
References
Since most of you posted *nix or Windows specific references I will summarize here the best ones, I will mark as accepted answer the Wikipedia one, because is a good start point (and has references inside too) to get introduced to this stuff.
Program Library Howto (Unix)
Dynamic-Link Libraries (from MSDN) (Windows)
DLL Information (StackOverflow) (Windows)
Programming in C (Unix)
An Overview of Compiling and Linking (Windows)
Start with Wikipedia - plenty of information there, and lots of links to other useful resources.
P.S. But perhaps it would be better to just ask a specific question about the problem you're currently having. Learning how to solve it may go a long way to teaching you the general concepts.
You can find some background information from this article here. It gives you the basic background. I'm trying to locate something with diagrams. This should be a good place to get started.
The fundamental differences between a static library and a DLL is that with the static library the code is compiled into your final executable whereas a dynamic link library involves linking in a "stub" library (into your application) which contains mappings to functions in a separate file (.dll).
Here's an MSDN entry on creating a static Win32 Library which might also help you.
..another link to MSDN for creating a Dynamic Link Library..
Just found this site which covers definitions of basically all the aspect you've quoted.
There is always MSDN for windows related stuff:
Head page for dlls ->
http://msdn.microsoft.com/en-us/library/ms682589
For Unix my favorite reference manual:
Programming in C, UNIX System Calls and Subroutines using C ->
http://www.cs.cf.ac.uk/Dave/C/
RM
See if these are useful:
Static and dynamic libraries
Program Library HOWTO