I am writing a C++ package with a number of optional dependencies. It is important to be able to use the package without any of these dependencies and then be able to add dependencies without recompiling the software. If a dependency is not available, I want to fall back to some default behaviour.
Each optional dependency corresponds to a different shared library of the package and I was thinking about (ab)using the “soversions” to implement this feature and have the minor revision 0 correspond to a fallback implementation.
To illustrate, assume that there is a core module compiled into a shared library called “libmy_core.so”. I have one optional feature (depending on “libmy_core.so” as well as some external package) compiled into the shared library “libmy_feature.so.1.1”. But to make sure the code works without this library, I also build a fallback module called “libmy_feature.so.1.0” which only depends on “libmy_core.so”.
I will then distribute “libmy_core.so” and “libmy_feature.so.1.0”. If the user later adds the optional dependeny, “libmy_feature.so.1.1” will also be installed (and take precedence over "libmy_feature.so.1.0”).
Is this a feasible solution or is there a better solution? Will a similar approach also work on non-Linux systems, in particular OS X and Windows?
Is this a feasible solution or is there a better solution?
This is not a good solution. You are setting yourself and your users up for dependency hell. Below is an alternative that can work. It is related to the idea of plugin libraries.
You can build the optional features in a shared library that exposes a C interface. Your application/library can then check for the precence of the shared library using dlopen, then get the interface using dlsym. If the shared library is not found or doesn't expose the required interface, your application/library can use the fallback behavior, which is implemented in the application/library itself. You have to be careful that your application/library does not depend on the optional library, but instead only uses the function pointers returned by dlsym.
The reason to use a C interface is because it provides a stable binary interface. With C++ types you can get undefined behavior when mixing shared libraries compiled with different compilers or using different compiler settings.
On Windows you can do the same using LoadLibrary and GetProcAddress.
I would load the optional library with dlopen. Then you just need one library and can use the dlopen return value to detect the other library at runtime.
Related
I have a question related to embedding one library in another.
I have a code that is pure C and my users rely on that, they don't want to depend on C++ libraries. However, the need arose to embed a 3rd party library (ICU) into mine. None of the ICU functions would be exported, they would only be used internally in my library. Unfortunately ICU is a C++ library, though it does have a C wrapper. ICU does not use exceptions, but it does use RTTI (abstract base classes).
The question is how could I create my static library so that
ICU is embedded in my library (all references to ICU functions are resolved within my library)
all references to libstdc++ are also resolved and the necessary code is embedded into my library
if a user does not even have libstdc++ installed on their system things work just fine
if a user does happen to use my library within a C++ project then there are no conflicts with whatever libstdc++ (presumably the system libstdc++) he uses.
Is this possible at all? The targeted platforms are pretty much everything: windows (there my library is dynamic), and all sort of unix versions (linux, solaris, aix, hpux - here my library needs to be static).
gcc-4.5 and later does have --static-libstdc++, but as far as I understand it is only for creating shared libs or executables, and not static libs.
Thanks for any help!
The solution to this problem is pretty simple, but may not fall inside the parameters you have set.
The simple rules are:
You can dynamically link a C++ library to a C caller, but you have to wrap the library inside an extern C layer. You design the extern C API and implement it using C++ internals, which are forever hidden from view. [You can also use COM or .NET to do this on Windows.]
You cannot statically link a C++ library to a C caller. The libraries are different and the calling sequences/linker symbols are different. [You often can't even statically link between different versions of the same compiler, and almost never between different compilers.]
In other words, the solution is simple: use dynamic linking. If that's not the right answer, then I don't think there is one.
Just to make things interesting, you can even implement your own plug-in architecture. That's just another name for dynamic linking, but you get to choose the API.
Just to be clear, the only viable portable option I can see is that you link ICU inside its own dynamic library (DLL or SO). Its symbols, C++ libs, RTTI and exceptions all stay inside that. Your static lib links to the ICU dynamic lib by extern C. That's exactly how much of Windows is built: C++ inside DLL, extern C.
You can debug across the boundary, but you cannot export type information. If you need to do that, you will have to use a different API, such as .NET or COM.
I don't know if this will work, but let me at least suggest that you try it!
The excellent LLVM project (origin of clang compiler) has many front-ends and back-ends for different languages, such as C++ and C. And according to this S.O. question it should be possible for LLVM to compile C++ into C which in turn can be compiled as normal.
I imagine this route is a bumpy one, but if it works, it might solve your problem without the need to link dynamically. It all depends on if ICU will compile with LLVM C++.
If you do decide to give it a go, please let us know how you fare!
I know this is not a very clean thing to do but how do I do it nonetheless?
Basically, I am statically linking a third party library that uses xerces-c 2.7 and I want to use xerces-c 3.1 (for some of the newer latest and greatest features not really available in 2.x)
The modules that use 2.7 (used internally by the third party library and never exposed to my code) have nothing in common with the modules using 3.1 (in my code).
Any way how to do this? I know it's not a good thing but I shudder to think of the lead time between submitting an upgrade request for the library and actually getting it done. Probably months at least and I don't want to go down that unholy path.
A generic compiler independent solution would ofc be much better.
Another solution aside from that mentioned in " Linking libraries with incompatible dependecies " is to isolate the different versions by building them into different dynamic libraries. The simplest approach may be to move the code that uses xerces 3.1 into a new dynamic library, and create an interface to it. Since you're statically linking against Xerces, this will keep the references internal to the dynamic library. You may need to change the gcc visibility settings to ensure that only selected function names are exported from the dynamic library.
I'm working on a cross-platform Firebreath plugin which is crashing on Windows. I use a static library containing classes which reference boost.asio. When I link this library against the plugin dll, I observe crashes when interacting with the io_service subsystem (ie, during socket construction). When I link the static library against an ordinary executable, the problem does not occur. When I compile the contents of the static library directly into the plugin dll project, the crash does not occur. I have gone to great lengths to ensure that all aspects of my build environment on Windows are consistent (build modes, version of Visual Studio, etc). In addition, I've firewalled the boost.asio headers, so that the plugin dll code has no visibility of the boost.asio sub-system (to no effect, unfortunately, vs2008 and vs2010). As far as I can tell, I've done everything possible to ensure that the build environment is well behaved but the problem persists.
Can the community offer any advice on potential risks or approaches which could expose or solve the problem?
Two things that are dramatically different between linking a static library vs loading a DLL:
global initialization: In a DLL, they all run. In a static library, the linker only brings in an object file if it satisfies some unresolved external, so systems which rely on components registering themselves using the constructor or intiialization expression of a global object fail.
shared CRT: In a static library, all calls to the Standard library are resolved during link time, and the main application and library function all call the same copy of the Standard library. In a DLL, you risk having two copies of the Standard library, which can be ok if you're careful never to share any Standard library objects (like FILE*, std::string, or even malloc/free pairs) between library and application.
The second thing is most likely what's biting you. There's a lazy way to fix it: Use the Standard library DLL, and a better way of fixing it: figure out memory and object lifetime and don't try to allocate on one side and free on the other, or share C++ class layout across the boundary. Virtual functions work fine across the boundary.
The advantage of the "better" way is that plugins can be built in any version of the compiler, which becomes a big deal for maintenance later on in the development cycle.
In the FireBreath plugin prep script, try turning on the WITH_DYNAMIC_MSVC_RUNTIME flag.
My (C++, cross-platform) app is heavily using Boost libraries (say version 1.x), and I want to also link against a 3rd-party (vendor)'s SDK (no source), itself using Boost (but version 1.y).
So, we both link dynamically against our own version of Boost DLLs, CRT being identical. Consequently, at run-time my app would have to load both DLL of Boost 1.x & 1.y.
What are the potential issues & gotchas associated?
I can't change vendor's SDK, but I can change my app. Maybe I should try to link statically against my Boost 1.x?
PS: Name of Boost's DLL include their version, so no name collision, both are identifiable. Not the usual DLL-hell.
As far as using the DLLs for different versions there should be no problem. At least not on Windows.
This is true if the SDK is using boost internally. If the SDK uses boost constructs in its interface, for example: it has a function that returns a boost::optional, then having multiple versions can cause problems. It could still work fine, dependent on the changes between the versions, but that will definitely be a risk. I don't know of any good solution in that case. This is also true if you include a SDK header file that includes a boost header file.
This is a big problem.
Do a search on DLL hell.
Basically the DLL (or shared libs in Linux) are loaded but not all the names are resolved at load time. What happens is a lazy evaluation, so the names are evaluated on first use. The problem is that if 2 dll have the same name then the location where the name is resolved to depends on the what order the DLL are searched in (which depends on load order).
If you statically link then you will not have problems with method calls as yours will all be resolved at compile time and the third party will be resolved at runtime from the DLL. But what about structures that are created by version-1 boost. If you then pass these to the third party library that then passes it to the version-x boost. Are the structures layed out in the same way?
This is a very tricky area and when problems occur very hard to de-bug.
So try and use the same version.
If you write a function foo, and export it from F.dll, and another function foo exported from G.dll, would you expect problems?
When AF.exe is linked, the linker is told: put some code in there that loads the address of function foo from F.dll. Now BG.dll is linked to retrieve the foo address from G.dll. I still see no problem.
Now replace AF.exe with your app, BG.dll with your vendor's app, F.dll with your boost version, G.dll with the vendor's boost version.
Concluding: I see no problems if the dll names are different.
I'm working on a certain C++ library (or more framework). I want to make it backward
compatible with previous versions preserving not only API compatibility but also ABI (like the great job Qt does).
I use lots of functionality of Boost and it seems for me that this makes backward compatibility just impossible, unless I force a user to have exactly the same (sometimes old) version of Boost.
Is there any way (without rewriting 1/2 of Boost) to make some "prefix" around its namespace/rename it in order to prevent it from interfering with a user version of Boost?
For example my libXYZ uses Boost 1.33 and it has class boost::foo. In version 1.35 boost::foo was upgraded and new member was added, so, boost::foo from 1.33 and 1.35 are
not ABI compatible. So, a user of libXYZ must use Boost 1.33 or recompile libXYZ with
Boost 1.35 (that may be already had broken some API in a way XYZ would not compile).
Note: I'm talking about UNIX/Linux OS with ELF where dynamic linking is similar to static linking, so you can't link with two different versions of libraries because symbols would interfere.
One suitable solution I may think of is putting Boost in some other private namespace. So, libXYZ would use ::XYZ::boost::foo instead of ::boost::foo. This would prevent collision with other version of Boost that user may use.
So, the libXYZ would continue to work with Boost 1.33 statically or dynamically linked with it withing other namespace, assuming, that it:
Would not expose Boost API outside.
Would keep stable private version of exposed API.
Is there any way to do such things with Boost?
Edit: Finally I decided to create a script that would rename all boost symbols in the source to some custom symbol.
Rationale: simplification of build process, independent of compiler visibility support, also, it visibility works on dynamic libraries only, for static this does not work, so I need separate builds and dependencies for each type of libraries.
The script is available there: http://art-blog.no-ip.info/files/rename.py
Edit 2: Latest version of Boost BCP supports namespace renaming.
Basically, just make sure the public interface to your library does not expose Boost. You can always use it however much you want internally. Generally, having the interface of a library depend on another library is bad (unless it depends on a standard library like STL). Boost almost fits into the "standard" library category, but its ABI changes so much that that your interface shouldn't use it.
To make sure you don't expose the Boost symbols, there are a few things you could do:
A. compile with -fvisibility=hidden and mark all public symbols with __attribute__((visibility("default"))). you could use a macro to make this easier:
#define ABI __attribute__((visibility("default")))
B. do something like this:
#pragma GCC visibility push(hidden)
#include <boost/whatever.hpp>
#pragma GCC visibility pop
You should also wrap this around all other internal symbols that you don't want exported, or declare this with __attribute__((visibility("hidden"))). Again, you could use a macro to make this easier:
#define INTERNAL __attribute__((visibility("hidden")))
Of these options, I like A better, because it makes you explicitly think about which symbols are exported, so you don't accidentally export things you don't want.
By the way, you can find a lot more information about making DSOs in Ulrich Drepper's How to Write Shared Libraries.
In general you can't rely on any type of ABI in C++ beyond standard C bindings. But Depending on how many assumptions you make, you can use more and more of C++ in your interface.
I found this great article on steps to make your API's turn into a stable ABI. For instance, never pass Standard C++ Library (or Boost) datatypes across your interface; it might break even with a small bug fix to the library.
Some examples of the issues to watch out for when publishing an ABI compatible API are:
Windows Debug heap. You have to be sure that all allocations and deallocations are on the same side of a "module" (that is, executable or DLL).
Fragile Binary Interface problem. Even if both sides of your system consistently use the same compiler and libraries, you have to be careful in C++ about what you publish in your .h files, and where allocations happen.
If you follow the linked article, you will find solutions for these and other issues.
Edit:
I also found an interesting article published by Microsoft which describes how COM interfaces work, by taking a C++ project and turning it into COM. It's my belief that one of the primary reasons that Microsoft developed COM was to solve the Fragile Binary Interface problem that C++ has, so they can ship DLLs with publish object oriented APIs.
Consider using abi-compliance-checker tool to maintain a stable API/ABI interface.
You should be able to do something like this:
namespace XYZ
{
#include <boost/my_library.hpp>
}
And it should dump the boost headers into namespace XYZ. Note, however, that this will only work with the header-only libraries.