I have a question related to embedding one library in another.
I have a code that is pure C and my users rely on that, they don't want to depend on C++ libraries. However, the need arose to embed a 3rd party library (ICU) into mine. None of the ICU functions would be exported, they would only be used internally in my library. Unfortunately ICU is a C++ library, though it does have a C wrapper. ICU does not use exceptions, but it does use RTTI (abstract base classes).
The question is how could I create my static library so that
ICU is embedded in my library (all references to ICU functions are resolved within my library)
all references to libstdc++ are also resolved and the necessary code is embedded into my library
if a user does not even have libstdc++ installed on their system things work just fine
if a user does happen to use my library within a C++ project then there are no conflicts with whatever libstdc++ (presumably the system libstdc++) he uses.
Is this possible at all? The targeted platforms are pretty much everything: windows (there my library is dynamic), and all sort of unix versions (linux, solaris, aix, hpux - here my library needs to be static).
gcc-4.5 and later does have --static-libstdc++, but as far as I understand it is only for creating shared libs or executables, and not static libs.
Thanks for any help!
The solution to this problem is pretty simple, but may not fall inside the parameters you have set.
The simple rules are:
You can dynamically link a C++ library to a C caller, but you have to wrap the library inside an extern C layer. You design the extern C API and implement it using C++ internals, which are forever hidden from view. [You can also use COM or .NET to do this on Windows.]
You cannot statically link a C++ library to a C caller. The libraries are different and the calling sequences/linker symbols are different. [You often can't even statically link between different versions of the same compiler, and almost never between different compilers.]
In other words, the solution is simple: use dynamic linking. If that's not the right answer, then I don't think there is one.
Just to make things interesting, you can even implement your own plug-in architecture. That's just another name for dynamic linking, but you get to choose the API.
Just to be clear, the only viable portable option I can see is that you link ICU inside its own dynamic library (DLL or SO). Its symbols, C++ libs, RTTI and exceptions all stay inside that. Your static lib links to the ICU dynamic lib by extern C. That's exactly how much of Windows is built: C++ inside DLL, extern C.
You can debug across the boundary, but you cannot export type information. If you need to do that, you will have to use a different API, such as .NET or COM.
I don't know if this will work, but let me at least suggest that you try it!
The excellent LLVM project (origin of clang compiler) has many front-ends and back-ends for different languages, such as C++ and C. And according to this S.O. question it should be possible for LLVM to compile C++ into C which in turn can be compiled as normal.
I imagine this route is a bumpy one, but if it works, it might solve your problem without the need to link dynamically. It all depends on if ICU will compile with LLVM C++.
If you do decide to give it a go, please let us know how you fare!
Related
I have currently the problem that i need to build a application where it seems that the libraries are build with different c++ versions.
When i build i got the undefined reference std::__cxx11::basic_string errors at one library and when i build with the -D_GLIBCXX_USE_CXX11_ABI=0 i get undefined reference errors at std::string of the other library.
They are both external libraries which i unfortunatly can't rebuild myself. Is there a workaround for this?
The only possible work-around I can think of is not very nice. Say you have libraries A and B with different ABI versions, vA and vB. Now, create a wrapper for library A which exposes an interface which is ABI-independent (plain C, i.e. all exposed functions should be declared with extern "C"). Then build this with ABI version vA (make sure to statically link needed vA libraries). Now you can build your application using ABI version vB linking with the wrapped library A (which should now be ABI-independent) and library B.
Of course, depending on the interface of library A, it may take some creativity to create a plain C interface, but it should be possible.
Disclaimer: I have not tried this myself, I recommend starting with a minimal test.
You can't mix different values of GLIBCXX_USE_CXX11_ABI in a single file but you should be able to link files with different settings into a single library.
So if you are using the c++11 library in class A and the c++03 library in class B and class A and class B are in separate files and don't have std::string or std::list in their headers then it should be possible to link class A and class B into a single library.
Use of the pimpl pattern may be useful/required to achieve this.
The simplest approach is probably to produce a static library which wraps your c++03 library and is compiled with -D_GLIBCXX_USE_CXX11_ABI=0 then you can compile the rest of your code as normal as long as you only interface with your c++03 library through the static library.
I need some help with wrapping C++ libraries in XCode.
What I want to achieve is to create new library in XCode, import C++ library (I have .a and .h files), wrap it to Obj-C so I can import that library to MonoTouch.
The reason why I do it round way is that when I try to import C++ lib into MonoTouch, because of name mangling I keep getting WrongEntryPoint exceptions. Correct me if I'm wrong but there is no way for me to find out mangled names, which depends on compiler.
Thank you in advance
Correct me if I'm wrong but there is no way for me to find out mangled names, which depends on compiler.
Technically you could. Many compilers share the same mangling syntax, maybe the most useful and long-lasting gift from Itanium ;-)
However it will bring it's own pain (e.g. non-primitive types, other compilers) and maintenance issues as you update your C++ code.
You'll better served by:
writing an ObjectiveC wrapper and use MonoTouch's btouch tool to generated bindings;
writing a C wrapper and using .NET p/invoke to call your code;
The choice it yours but if you think about reusing the C++/C# code elsewhere (e.g. Mono for Android) then using C and p/invoke will be reusable.
I would definitely recommend going the route of wrapping the library in an Obj-C library and using btouch to import the library into MonoTouch. I have recently done this for a C++ library that implemented a Sybase database engine. If you look at my questions you will find quite a few pertaining to wrapping C++ libraries as I posted a few times regarding issues I encountered.
Specifically, you can look at these questions:
Linking to a C++ native library in MonoTouch
Wrapping a C++ library in Objective-C is not hiding the C++ symbols
Application with static library runs on simulator but not on actual device
Undefined symbols when linking PhoneGap static library in MonoTouch
Linker options 'Link all assemblies" and "Link SDK assemblies only" causes undefined symbols in 3rd party static library
I would also recommend, if you are going to go the route of an Obj-C wrapper, that you get btouch to output code and include that in your project rather than including a dll from btouch. From my experience, the code worked more reliably than the dll, although the issues with the dll may have been resolved by now. But take a look at this question regarding the btouch issue:
Exception System.InvalidCastException when calling a method bound with btouch that returns an object. MonoTouch bug?
If you have specific questions/problems in building the Obj-C wrapper then ask them here and post some code and I am sure that I or other members of the community would be able to help you with it.
Bruce, as you assumed, I have problems with wrapping C++ code. After hours and hours of reading and trying, I couldn't wrap the C++ code.
Anyway, I managed to create a simple Obj-C library made of some dummy class, and then import it into another library. That worked fine. However, following same pattern, I included C++ .a file along with .h file (I'm not sure whether .h is mandatory because we can link header files in build options, right??) and when I compiled it, it went fine, the build succeeded, but XCode didn't produce new .a library.
I added linker flags: -ObjC -lNameOfLib
Does C++ Standard Library Type in Build - Linking has to be Static? And Symbols Hidden By Default as well?
It would be great if we could write step-by-step tut, since there are tons of various instructions, but I haven't been able to push it through the end.
I'm confused a bit..
Thank you guys...
I've developed a module written in C++ that manages the licenses for my company's product. To prevent DLL replacement, it is our goal to statically link the DLL in the solution. This is very easy to do in C++ but proving to be a bit problematic for part of our codebase that is written in Fortran.
I realize that this could possibly vary from compiler to compiler (We use Intel Fortran 9.1), but is there any universal way to implement static linking of a C++ DLL within Fortran?
To get static linking, the usual way is not to use DLL but simple libraries instead (*.lib). This has nothing to do with programming languages : it just depends on the operating system.
Building a library is also simpler than building a DLL. On Unix, a library has the suffix .a whereas a DLL has a suffix .so (for shared object).
Nevertheless, it is often possible to link a DLL statically but this is obtained by a specific option passed to the linker. For instance on Unix, with many compiler suites, the option is either -static or -Bstatic. Look at the keyword "static" in your programming manual of your compilers.
If you have access to the source, just compile it to object files and link them into your Fortran project. ISO_C_BINDING should work on many compilers.
I'm working on a certain C++ library (or more framework). I want to make it backward
compatible with previous versions preserving not only API compatibility but also ABI (like the great job Qt does).
I use lots of functionality of Boost and it seems for me that this makes backward compatibility just impossible, unless I force a user to have exactly the same (sometimes old) version of Boost.
Is there any way (without rewriting 1/2 of Boost) to make some "prefix" around its namespace/rename it in order to prevent it from interfering with a user version of Boost?
For example my libXYZ uses Boost 1.33 and it has class boost::foo. In version 1.35 boost::foo was upgraded and new member was added, so, boost::foo from 1.33 and 1.35 are
not ABI compatible. So, a user of libXYZ must use Boost 1.33 or recompile libXYZ with
Boost 1.35 (that may be already had broken some API in a way XYZ would not compile).
Note: I'm talking about UNIX/Linux OS with ELF where dynamic linking is similar to static linking, so you can't link with two different versions of libraries because symbols would interfere.
One suitable solution I may think of is putting Boost in some other private namespace. So, libXYZ would use ::XYZ::boost::foo instead of ::boost::foo. This would prevent collision with other version of Boost that user may use.
So, the libXYZ would continue to work with Boost 1.33 statically or dynamically linked with it withing other namespace, assuming, that it:
Would not expose Boost API outside.
Would keep stable private version of exposed API.
Is there any way to do such things with Boost?
Edit: Finally I decided to create a script that would rename all boost symbols in the source to some custom symbol.
Rationale: simplification of build process, independent of compiler visibility support, also, it visibility works on dynamic libraries only, for static this does not work, so I need separate builds and dependencies for each type of libraries.
The script is available there: http://art-blog.no-ip.info/files/rename.py
Edit 2: Latest version of Boost BCP supports namespace renaming.
Basically, just make sure the public interface to your library does not expose Boost. You can always use it however much you want internally. Generally, having the interface of a library depend on another library is bad (unless it depends on a standard library like STL). Boost almost fits into the "standard" library category, but its ABI changes so much that that your interface shouldn't use it.
To make sure you don't expose the Boost symbols, there are a few things you could do:
A. compile with -fvisibility=hidden and mark all public symbols with __attribute__((visibility("default"))). you could use a macro to make this easier:
#define ABI __attribute__((visibility("default")))
B. do something like this:
#pragma GCC visibility push(hidden)
#include <boost/whatever.hpp>
#pragma GCC visibility pop
You should also wrap this around all other internal symbols that you don't want exported, or declare this with __attribute__((visibility("hidden"))). Again, you could use a macro to make this easier:
#define INTERNAL __attribute__((visibility("hidden")))
Of these options, I like A better, because it makes you explicitly think about which symbols are exported, so you don't accidentally export things you don't want.
By the way, you can find a lot more information about making DSOs in Ulrich Drepper's How to Write Shared Libraries.
In general you can't rely on any type of ABI in C++ beyond standard C bindings. But Depending on how many assumptions you make, you can use more and more of C++ in your interface.
I found this great article on steps to make your API's turn into a stable ABI. For instance, never pass Standard C++ Library (or Boost) datatypes across your interface; it might break even with a small bug fix to the library.
Some examples of the issues to watch out for when publishing an ABI compatible API are:
Windows Debug heap. You have to be sure that all allocations and deallocations are on the same side of a "module" (that is, executable or DLL).
Fragile Binary Interface problem. Even if both sides of your system consistently use the same compiler and libraries, you have to be careful in C++ about what you publish in your .h files, and where allocations happen.
If you follow the linked article, you will find solutions for these and other issues.
Edit:
I also found an interesting article published by Microsoft which describes how COM interfaces work, by taking a C++ project and turning it into COM. It's my belief that one of the primary reasons that Microsoft developed COM was to solve the Fragile Binary Interface problem that C++ has, so they can ship DLLs with publish object oriented APIs.
Consider using abi-compliance-checker tool to maintain a stable API/ABI interface.
You should be able to do something like this:
namespace XYZ
{
#include <boost/my_library.hpp>
}
And it should dump the boost headers into namespace XYZ. Note, however, that this will only work with the header-only libraries.
Over the months I've written some nice generic enough functionality that I want to build as a library and link dynamically against rather than importing 50-odd header/source files.
The project is maintained in Xcode and Dev-C++ (I do understand that I might have to go command line to do what I want) and have to link against OpenGL and SDL (dynamically in SDL's case). Target platforms are Windows and OS X.
What am I looking at at all?
What will be the entry point of my
library if it needs one?
What do I have to change in my code?
(calling conventions?)
How do I release it? My understanding
is that headers and the compiled
library (.dll, .dylib(, .framework),
whatever it'll be) need to be
available for the project -
especially as template functionality
can not be included in the library by
nature.
What else I need to be aware of?
I'd recommend building as a statc library rather than a DLL. A lot of the issues of exporting C++ functions and classes go away if you do this, provided you only intend to link with code produced by the same compiler you built the library with.
Building a static library is very easy as it is just an collection of .o/.obj files - a bit like a ZIP file but without compression. There is no need to export anything - just include the library in the list of files that your application links with. To access specific functions or classes, just include the relevant header file. Note you can't get rid of header files - the C++ compilation model, particularly for templates, depends on them.
It can be problematic to export a C++ class library from a dynamic library, but it is possible.
You need to mark each function to be exported from the DLL (syntax depends on the compiler). I'm poking around to see if I can find how to do this from xcode. In VC it's __declspec(dllexport) and in CodeWarrior it's #pragma export on/#pragma export off.
This is perfectly reasonable if you are only using your binary in-house. However, one issue is that C++ methods are named differently by different compilers. This means that nobody who uses a different compiler will be able to use your DLL, unless you are only exporting C functions.
Also, you need to make sure the calling conventions match in the DLL and the DLL's client. This either means you should have the same default calling convention flag passed to the compiler for both the DLL or the client, or better, explicitly set the calling convention on each exported function in the DLL, so that it won't matter what the default is for the client.
This article explains the naming issue:
http://en.wikipedia.org/wiki/Name_decoration
The C++ standard doesn't define a standard ABI, and that's bad news for people trying to build C++ libraries. This means that you get different behavior from your compiled code depending on which flags were used to compile it, and that can lead to mysterious bugs in code that compiles and links just fine.
This extends beyond just different calling conventions - C++ code can be compiled to support or not support RTTI, exception handling, and with various optimizations that can affect the the memory layout of class instances, which C++ code relies on.
So, what can you do? I would build C++ libraries inside my source tree, and make sure that they're built as part of my project's build, and that all the libraries and the code that links to them use the same compiler flags.
Note that name mangling, which was supposed to at least prevent you from linking object files that were compiled with different compilers/compiler flags only mostly works, and there are certain things you can do, especially with GCC, that will result in code that links just fine and fails at runtime.
You have to be extra careful with vendor supplied dynamic C++ libraries (QT on most Linux distributions, for example.) I've seen instances of vendor supplied libraries that were compiled in ways that prevented certain things from working properly. For example, some Redhat Linux releases (maybe all of them) disabled exceptions in QT, which made it impossible to catch exceptions in main() if the exceptions were thrown in a QT callback. Fun.