In my project there is a dependency on a static library (just called libsomething from now on) from a 3rd party. Recently, libsomething has become available in another version. My task is to provide my software with support for the old and the new version. Only one version of libsomething is used at run-time at any given time, but which version this is should be configurable between program runs.
I am using MSVC2005 on WinXP, a secondary objective is to become prepared to switch over to Linux and GCC.
Since both versions of libsomething are using the same symbols, linking them both into my executable is out of the question as the symbols of both versions are going to clash all over at link-time.
While I could create two executables (one linking against the old version, the other one using the new version), I cannot implement a decision on which executable to call in the final deployment environment (legacy reasons).
I came up with the idea of creating a dynamic library wrapper for each version of libsomething and linking them at run-time depending on some config file. With MSCV, this would mean going down the road of using LoadLibrary(), GetProcAddress(), etc., while on Linux I would have to use dlopen() and dlsym().
I understand that using libtool (i.e., libtldl) is wrapping this platform-dependency for using shared libraries. Is this an appropriate path to follow? Are there better (or, at least, different) ways? Do alternatives for libtldl exist as open-source?
I know you said you couldn't use two executables due to the decision of which to execute, but couldn't you exec back and forth between executables depending on which version is selected at configuration?
On Linux it would be easier for you to link to shared library and use symlinks to correct version - IMO it's much easier than using dlopen() + dlsym().
Thus you would create shared libraries for the old and new versions of your library:
g++ -shared -o libshared.so.1.1 -Wl,-soname=libshared.so.1 -L. -Wl,--whole-archive libstatic.a.old -Wl,-no-whole-archive
and
g++ -shared -o libshared.so.1.2 -Wl,-soname=libshared.so.1 -L. -Wl,--whole-archive libstatic.a.new -Wl,-no-whole-archive
Create the symlinks:
ln -s libshared.so.1.1 libshared.so.1
ln -s libshared.so.1 libshared.so
Build your application, linking it to the old version of the library. I suppose both versions are binary compatible (ABI not broken), but the new one could have some new symbols.
g++ -o myapp myapp.cpp -L. -lshared
Since shared library's SONAME is libshared.so.1 your application will depend on it and will search libshared.so.1 in paths from /etc/ld.so.conf or LD_LIBRARY_PATH
Before you run your application you may set the libshared.so.1 symlink to point to libshared.so.1.2 or libshared.so.1.1.
Little info about the linker options used here:
--whole-archive
For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive in the
link, rather than searching the archive for the required object files. This is normally used to turn an archive file into a shared
library, forcing every object to be included in the resulting shared library. This option may be used more than once.
Two notes when using this option from gcc: First, gcc doesn't know about this option, so you have to use -Wl,-whole-archive.
Second, don't forget to use -Wl,-no-whole-archive after your list of archives, because gcc will add its own list of archives to your
link and you may not want this flag to affect those as well.
-soname=name
When creating an ELF shared object, set the internal DT_SONAME field to the specified name. When an executable is linked with a
shared object which has a DT_SONAME field, then when the executable is run the dynamic linker will attempt to load the shared object
specified by the DT_SONAME field rather than the using the file name given to the linker.
It has been few years now but I'd like to mention another solution for completeness. Instead of manual dlopen and dlsym you could generate simple stubs for all necessary functions and on first call (or at program startup) decide which library version is needed, load it and resolve the addresses.
You could write a script specifically tailored for your project or use Implib.so tool:
# This will generate mylib.so.init.c and mylib.so.tramp.S
# which implement stubs. These need to be linked to your
# executable.
$ implib-gen.py mylib.so
Implib.so is Linux-only atm but should be easily adaptable to Windows.
Related
In order to make use of C++11 and c++14 features I have an application compiled using
a newer version of gcc (4.9.1) and thus an newer version of libstdc++. The application consists of many
small programs so I am linking with libstdc++ as a shared library rather than a static one
(i.e. I don't wish to use -static-libstdc++)
I wish to ship the new version of libstdc++ with the application under /opt//lib64
(note: that this is specifically allowed under an exception to the GPL)
The new version of libstdc++.so differs with the version on the target platform only by minor version.
libstdc++ is designed to be forward compatible so that existing programs can use the new version of the library. However, I have observed subtle differences in behaviour (i.e. bugs) when some
programs use the new version rather than the older one. I wish to prevent this.
I also find that ld will try to link my application with the system version of libstdc++ unless
I put /opt//lib64 earlier on the LD_LIBRARY_PATH.
Supposedly you can force linking against a specific version using -l:<library>.<version>, however, this doesn't seem to work. I suspect that it would for a user created library but not for
a language runtime library like libstd++ because gcc itself generates the linker script.
Also on one of my target platforms (RHEL5) it is not even understood by gcc/ld.
I presume this is possible via using -nostdlib and linking in everything required (e.g. -lgcc) in my build system
rather than leaving it to gcc which I would prefer. So far I haven't tried this.
A simple way workaround for this is to ensure LD_LIBRARY_PATH contains /opt//lib64 when I run my application
and not otherwise or equally I could use LD_PRELOAD with the correct library version.
The problem with this if someone decides ignore my advice and runs
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/<vendor>/lib64
it could lead to subtle and hard to diagnose problems. So I have been looking for a better way.
I wondered if there was some way I could rename libstdc++ as lib_stdc++ and link against that soname instead.
Renaming libstdc++ is not enough as you need to alter the soname in the file as given by readelf. i.e
0x000000000000000e (SONAME) Library soname: [libstdc++.so.6]
If you do this on a normal program linked by gcc even using -l:stdc++.so.6.0.20 say you will notice this
gives the major version and not the specific minor version.
i.e.
readelf -d <myapp>
0x0000000e (SONAME) Library soname: [libstdc++.so.6]
rather than:
0x0000000e (SONAME) Library soname: [libstdc++.so.6.0.20]
So I have instead created a dummy shared library with the soname I want to depend on in order to add a dependency as follows:
gcc dummy.o -Wl,-soname,lib<vendor>_stdc++.so.6.0.20 -nostdlib -shared -o lib<vender>_dummycpp.so
(where dummy.o is an empty object file made from an empty source file to stop -nostdlib leading to complaints)
then:
gcc <myapp> -l<vendor>_dummycpp
as desired I now get:
readelf -d <myapp>
0x0000000000000001 (NEEDED) Shared library: [lib<vendor>_stdc++.so.6.0.20]
instead of
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
The dummycpp library contains all the symbols in libstdc++ because it is libstd++ so there is no need for gcc to link with the soname libstdc++ at all. I appear to have solved the issue completely.
It strikes me as a slightly hacky solution so the questions I want to ask here are:
Is this a good idea?
If not why not?
Is there a better/more correct way?
Note: that if you package the libstdc++ with a different name (e.g. lib_stdc++.so.6.0.20) in an RPM you will get a missing dependency on it and need to install using --nodeps. This is because RPM scans for link time dependencies. The workaround for this is to install the dummy library as well. RPM will then pick up the soname from the dummy library and not claim it is missing.
The libstdc++ FAQ entry How do I insure that the dynamically linked library will be found links to the manual section Finding Dynamic or Shared Libraries which explains how to use RPATH instead.
My preferred method is to use an RPATH of $ORIGIN which means that searching for dynamic library dependencies starts in the same directory as the binary (see ld.so(8)). So if you link with '-Wl,-rpath,$ORIGIN' (note the quotes to prevent $ORIGIN being expanded by the shell) then you can install shared libraries in the same directory as your installed binary and they will be found when your binary is run. Or use '-Wl,-rpath,$ORIGIN/../lib' if you'd rather have separate bin and lib directories under some installation prefix.
With the library installed alongside the binary in some custom path that ldconfig doesn't scan, and no LD_LIBRARY_PATH messing up the environment, the newer libstdc++ will never be found by applications which are not supposed to use that version.
Make sure you also install the libstdc++.so.6 symlink pointing to the libstdc++.so.6.0.20 file so that the DT_NEEDED for libstdc++.so.6 can find the file.
I have a C++ file a.cpp with the library dependency in the path /home/name/lib and the name of the library abc.so.
I do the compilation as follows:
g++ a.cpp -L/home/name/lib -labc
This compiles the program with no errors.
However while running the program, I get the ERROR:
./a.out: error while loading shared libraries: libabc.so.0: cannot open shared object file: No such file or directory
However if before running the program, I add the library path as
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/name/lib;
and compile and run now, it works fine.
Why am I not able to link the library by giving it from the g++ command?
Because shared object libraries are linked at runtime - you either need to add the location to the library search path (as you've done), place it somewhere within an already existing path (like /usr/lib), or add a symbolic link to an existing library path that links to the location where it exists.
If you want to compile the library in so there is no runtime dependency, you'll need a static library (which would be abc.a) - note that this has numerous downsides, however (if the library is updated, you'll need to recompile your executable to incorporate that update).
Why am I not able to link the library by giving it from the g++ command?
You are able to link, and you did link the library succesfully. Otherwise you would not be able to build executable (in your case a.out). The problem you mixed 2 different things: linking with shared libraries and loading them at runtime. Loading shared libraries is a pretty complex concept and described pretty well here Program-Library-HOWTO read from 3.2.
You are linking dynamically, is the default behavior with GCC. LD_LIBRARY_PATH is used to specify directories where to look for libraries (is a way of enforce using an specific library), read: Program-Library-HOWTO for more info. There is also an ld option -rpath to specify libraries search path for the binary being compiled (this info is written in the binary and only used for that binary, the LD_LIBRARY_PATH affect other apps using the same library, probably expecting a new or old version).
Linking statically is possible (but a little tricky) and no dependency would be required (but sometimes is not recommended, because prevent the update of the dependent libraries, for example for security reason, in static linking your always are using the versions of the libraries you have when compiled the binary).
I have downloaded the OpenSSL binary file. I would like to create a static library for my C++ program in Ubuntu.
Meaning that they are in the same directory.
http://www.openssl.org/source/
Add -static parameter to gcc when you are linking. I expect you want to static binary without any dynamic loaded libraries. In other case, add full path to libssl.a as object file to linking in your build system. You have not specified how are you going to build your application.
Manually, you would use something like:
gcc -o application yourcode.c yourcode2.c /usr/lib/libssl.a
or better
gcc -static -o application yourcode.c yourcode2.c -lssl
Downloading binary for Linux is bad idea in most cases. If you want static binary, this should help. If you need custom build of library with special features, you need to download and build that library from sources yourself.
Anyway, similar question to yours is answered here at Static link of shared library function in gcc
You might also check Linux static linking is dead? to discover there are maybe too many problems to even consider static linking.
And if you need more information about linking under Linux, check nice tutorial at http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
For some reason I'm never able to use external libraries in any language. I'm looking for instructions/explanations of how to use external libraries, as well as how they work. When I search online, I get fragments that never seem to apply to whatever library I download and try and use. I work on both a mac and a pc, and C++ examples are fine. I use eclipse IDE with the C++ plug in. If there are instructions that apply to all libraries that would be great.
Say you have a class Unuseful defined as follows:
File Unuseful.h:
class Unuseful {
public:
void printUnusefulStatement();
};
File Unuseful.cpp:
#include "unuseful.h"
#include <iostream>
void Unuseful::printUnusefulStatement()
{
std::cout << "Hello world!" << std::endl;
}
Now, you have another class that needs printing unuseful statements:
Unuseful u;
u.printUnusefulStatement();
This means that you want to use an external library containing the specific implementation (printUnusefulStatement) that you want to include in your code.
You may use this library in two ways:
By providing the source code to the compiler
By providing a binary file (which had been previously compiled for your architecture), to the linker
Case 1: using a library at compile time
This is the simplest case.
You have the source code of the library you have to use and you simply have to compile it together with your existing code (say main.cpp file).
Typically you are the author and user of the library (a class that accomplishes a task you need).
Compiling with this command:
g++ main.cpp unuseful.cpp
allows you to use the implementation you need in your main.cpp file.
Case 2: linking a library
More often than Case 1, you don't have the source code of the library you want to use. You only have the header file (Unuseful.h, to continue with the example) and a static or shared library (probably[*] libunuseful.a and libunuseful.so files, respectively).
The static library is an archive of object files (*.o) that are linked inside your final executables, the shared libraries instead are loaded dynamically - at run time (look at this page for a better understanding of the difference).
Static libraries are created by simply archiving the *.o files with the ar program:
# Create the object files (only one here)
g++ -c unuseful.cpp
# Create the archive (insert the lib prefix)
ar rcs libunuseful.a unuseful.o
Shared libraries are created with the g++ -shared option:
# Create the object file with Position Independent Code[**]
g++ -fPIC -c unuseful.cpp
# Crate the shared library (insert the lib prefix)
g++ -shared -o libunuseful.so unuseful.o
Let's suppose now you have the Unuseful.h file and the shared library (libunuseful.so file) and you have a main.cpp file that instantiates a Unuseful object and calls the printUnusefulStatement method.
If you try to compile this file (g++ main.cpp) the linker will complain because it cannot find the printUnusefulStatement symbol.
It's time to use the library:
g++ main.cpp -L. -lunuseful
The -L option tells the linker where to search for library files and the -l flag tells the linker the name of the libraries to be used (without the lib prefix).
Now the executable (a.out, because I didn't specify a different name) is created, and you have used a library to implement a functionality you needed (printUnusefulStatement).
Since the shared library is loaded at run-time, the execution of the a.out executable may fail because the system is not able to find the library.
Typically this can be solved by appropriately setting an environment variable indicating which paths to use to search for dynamic libraries:
# Set the LD_LIBRARY_PATH [*]
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.
Done, now your executable has been compiled and it will be able to run and load the library it needs.
Conclusion
This is a rapid overview on libraries which I hope can help you understand how they are used and provided to others.
There are many many aspects that should be investigated in more detail, if you are interested: g++ options when creating shared libraries, ar options, environment variables, the shared libraries format and so on.
[*]: In a Unix environment
[**]: If supported for the target machine, emit position-independent code, suitable for dynamic linking and avoiding any limit on the size of the global offset table. This option makes a difference on the m68k, PowerPC and SPARC. Position-independent code requires special support, and therefore works only on certain machines. [From the g++ man page]
Here's where you start
http://en.wikipedia.org/wiki/Library_(computing)
Basically, a 'library' is a collection of compiled functions and class declarations.
On a Mac there are also "frameworks" which are somewhat similar to Pascal's units and contain both the declarations and the compiled code.
In managed languages like Java or C# there are packages and assemblies. Both are closely related to libraries.
To use libraries in C or C++ you've got to have a .lib-file (or .a-file for most POSIX or GCC toolchain based compilers) and the prototypes of the functions which are compiled into the .lib file. Depending on your development environment (for Eclipse you are most likely using the GCC compiler and GNU toolchain with LD linker), you just specify the library files (.lib or .a) as the input to the linker. Most of the time the library is accompanied with header files which contain the definitions of function prototypes.
Even if you did not know about the linker, which is strange enough, the libraries are still used in your program implicitly - the std::cout is in the libstdc++ or the C Run-Time Library.
As an example of a huge library and a useful set of C++ classes you might want to look at Boost.
To write GUI on Windows you can use the WinAPI which is described in MSDN.
To write GUI on Mac you can use Carbon API which is somewhat similar to WinAPI, but is now deprecated. The only way to write "legit" GUI for MacOS is to use Cocoa and Objective-C.
To write cross-platform GUI you can use a lot of libraries: Qt, wxWidgets, GTK among them.
The last, but not the least. C++ is not the best language for GUI.
The best way to use external C++ libraries is make use of a C++ package manager, go and learn one of these:
conan
vcpkg
hunter
cppan
build2
Some of them involve using CMake, you can find a well written tutorial on it here.
.
First, I don't know if there is a solution to my problem at all.
I have the following situation:
I have developed a framework library that depends on several other libraries for specific hardware access, etc.
Until now this framework library was only statically linked against.
For the executable that uses the framework library only the dependencies of code that is actually used by the executable have to be linked. (If I don't access a specific hardware at all I don't have to depend on its associated libraries.)
Now I need to also make a shared object of the framework library. Also the dependencies are available as shared libraries, so there is no need for any static linking.
The problem I have now:
When building an application that links dynamically to the framework library I have to either link all dependencies dynamically to the framework library or the application. (Otherwise I get undefined references complaints from ld)
My questions:
Is there any way to ignore certain shared object dependencies if I know that my application will not use any code of the framework library that depends on this shared object?
Is there any way to do this without or with minimal code changes? (linker / compiler switches)
I also need the static linking as described in the original situation to still work.
Additional Info:
Operating system: Linux (Debian Lenny)
Compiler: gcc-4.3
You can, but you basically have to do all of the dynamic library handling yourself. i.e. dlopen the library, and then look up the symbols you need directly with dlsym.
It will make your code more complicated, how much depends on the interface you've got into the libraries.
From man ld
--as-needed
--no-as-needed
This option affects ELF DT_NEEDED tags for dynamic libraries mentioned on the command line after the --as-needed option. Normally,
the linker will add a DT_NEEDED tag for each dynamic library mentioned on the command line, regardless of whether the library is
actually needed. --as-needed causes a DT_NEEDED tag to only be emitted for a library that satisfies a symbol reference from regular
objects which is undefined at the point that the library was linked, or, if the library is not found in the DT_NEEDED lists of other
libraries linked up to that point, a reference from another dynamic library. --no-as-needed restores the default behaviour.
I haven't used it myself but sounds like what you're looking for.
g++ -o your_app -Wl,--as-needed -lframework -la -lb -lc -Wl,--no-as-needed
Edit (suggested by Hanno)
--warn-unresolved-symbols
If the linker is going to report an unresolved symbol (see the option --unresolved-symbols) it will normally generate an error.
This option makes it generate a warning instead.