I am struggeling a little bit with some options for linking on a project I am currently working on:
I am trying to create a shared library which is linked against 2 other libraries. (Lets call them libfoo.so and libbar.so)
My output library has to be a shared library and I want to static link libfoo.so to the resulting library, but libbar.so should linked as a dynamic library. (libbar.so should be available on every machine, where libfoo.so is not available and I do not want the user install it / ship it with my binaries.)
How could I archive this?
My current build instruction look like this:
c++ -Wall -shared -c -o src/lib.o src/lib.cpp
c++ -Wall -shared -o lib.ndll src/lib.o -lfoo -lbar
I my defense: I am not a c/c++ expert, so sorry if this question seems to be stupid.
There are two Linux C/C++ library types.
Static libraries (*.a) are archives of object code which are linked with and becomes part of the application. They are created with and can be manipulated using the ar(1) command (i.e. ar -t libfoo.a will list the files in the library/archive).
Dynamically linked shared object libraries (*.so) can be used in two ways.
The shared object libraries can be dynamically linked at run time but statically aware. The libraries must be available during compile/link phase. The shared objects are not included into the binary executable but are tied to the execution.
The shared object libraries can be dynamically loaded/unloaded and linked during execution using the dynamic linking loader system functions.
In order to statically link libfoo.so into your binary, you will need a corresponding static library which is typically called libfoo.a. You can use a static library by invoking it as part of the compilation and linking process when creating a program executable.
The result would be changing your build commands to something like the following:
g++ -Wall -fPIC -c -o src/lib.o src/lib.cpp
g++ -shared -Wl,-soname,mylib.so.1 -o mylib.so.1 src/lib.o -L/path/to/library-directory -lbar libfoo.a
Related
I am creating a shared library from a class from an example I got here C++ Dynamic Shared Library on Linux. I would like to call another shared library from the shared library created and then use it in the main program. So I have the myclass.so library and I want to call another library say anotherclass.so from the myclass.so library and then use this myclass.so library in the main program. Any idea on how I can do this please.
There is more than one way in which multiple shared libraries may be added to
the linkage of a program, if you are building all the libraries, and the program,
yourself.
The elementary way is simply to explicitly add all of the libraries to the
the linkage of the program, and this is the usual way if you are building only the
program and linking libraries built by some other party.
If an object file foo.o in your linkage depends on a library libA.so, then
foo.o should precede libA.so in the linkage sequence. Likewise if libA.so
depends on libB.so then libA.so should precede libB.so. Here's an illustration.
We'll make a shared library libsquare.so from the files:
square.h
#ifndef SQUARE_H
#define SQUARE_H
double square(double d);
#endif
and
square.cpp
#include <square.h>
#include <cmath>
double square(double d)
{
return pow(d,2);
}
Notice that the function square calls pow, which is declared in the
Standard header <cmath> and defined in the math library, libm.
Compile the source file square.cpp to a position-independent object file
square.o:
$ g++ -Wall -fPIC -I. -c square.cpp
Then link square.o into a shared library libsquare.so:
$ g++ -shared -o libsquare.so square.o
Next we'll make another shared library libcube.so from these files:
cube.h
#ifndef CUBE_H
#define CUBE_H
double cube(double d);
#endif
and
cube.cpp
#include <cube.h>
#include <square.h>
double cube(double d)
{
return square(d) * d;
}
See that the function cube calls square, so libcube.so is going to
depend on libsquare.so. Build the library as before:
$ g++ -Wall -fPIC -I. -c cube.cpp
$ g++ -shared -o libcube.so cube.o
We haven't bothered to link libsquare with libcube, even though libcube
depends on libsquare, and even though we could have, since we're building libcube.
For that matter, we didn't bother to link libm with libsquare. By default the
linker will let us link a shared library containing undefined references, and it
is perfectly normal. It won't let us link a program with undefined references.
Finally let's make a program, using these libraries, from this file:
main.cpp
#include <cube.h>
#include <iostream>
int main()
{
std::cout << cube(3) << std::endl;
return 0;
}
First, compile that source file to main.o:
$ g++ -Wall -I. -c main.cpp
Then link main.o with all three required libraries, making sure to list
the linker inputs in dependency order: main.o, libcube.so, libsquare.so, libm.so:
$ g++ -o prog main.o -L. -lcube -lsquare -lm
libm is a system library so there's no need to tell the linker where to look for
it. But libcube and libsquare aren't, so we need to tell the linker to look for
them in the current directory (.), because that's where they are. -L. does that.
We've successfully linked ./prog, but:
$ ./prog
./prog: error while loading shared libraries: libcube.so: cannot open shared object file: No such file or directory
It doesn't run. That's because the runtime loader doesn't know where to find libcube.so (or libsquare.so, though it didn't get that far).
Normally, when we build shared libraries we then install them in one of the loader's default
search directories (the same ones as the linker's default search directories), where they're available to any program, so this wouldn't happen. But I'm not
going to install these toy libraries on my system, so as a workaround I'll prompt the loader where to look
for them by setting the LD_LIBRARY_PATH in my shell.
$ export LD_LIBRARY_PATH=.
$ ./prog
27
Good. 3 cubed = 27.
Another and better way to link a program with shared libraries that aren't located
in standard system library directories is to link the program using the linker's
-rpath=DIR option. This will write some information into the executable to tell
the loader that it should search for required shared libraries in DIR before it tries
the default places.
Let's relink ./prog that way (first deleting the LD_LIBRARY_PATH from the shell so that it's not effective any more):
$ unset LD_LIBRARY_PATH
$ g++ -o prog main.o -L. -lcube -lsquare -lm -Wl,-rpath=.
And rerun:
$ ./prog
27
To use -rpath with g++, prefix it with -Wl, because it's an option for linker, ld,
that the g++ frontend doesn't recognise: -Wl tells g++ just to pass the
option straight through to ld.
I would like to add some points to the response of #Mike.
As you do not link libcube library with libsquare you are creating a sort of "incomplete library". When I say incomplete, I meant that when you link your application you must link it with both libcube and libsquare even though it does not use any symbol directly from libsquare.
It is better to link libcube directly with libsquare. This link will create the library with a NEEDED entry like:
readelf -d libcube.so
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libsquare.so]
Then when you link your application you can do:
g++ -o prog main.o -L. -lcube
Although, this will not link because the linker tries to locate the NEEDED library libsquare. You must precise its path by adding -Wl,-rpath-link=. to the linking command:
g++ -o prog main.o -L. -lcube -Wl,-rpath-link=.
Note: For runtime, you must still set LD_LIBRARY_PATH or link with rpath as mentioned by #Mike.
In your library if you are using any other shared library so simply your library user is also dependent on that library. While creating library you can use -l so the linker have notion for shared library and it will link when required.
But when you deliver your library as its dependent on some other library you need to export that too along with your and provide some environment variable or linker flag to load it from specified path (Your exported package). That will not lead any discrepancy other wise if its some standard library function user might get definition from his system's some other library and will lead in disastrous situation.
Simply use the library like you'd use it in any other application. You don't have to link to anotherclass.so, just to myclass.so.
However, you will have to make both libraries (myclass.so and anotherclass.so) available for your later application's runtime. If one of them is missing you'll get runtime errors just like it is with any other application.
I'm working on a heavy project that has a lot of static libraries that are interdependent. Furthermore some symbols are redundant between some libraries, with different implementations. My goal is to make the project work with shared libraries.
I tried to compile an executable with one of my shared libs, and I get undefined symbols errors on functions that my executable isn't using. After some research I understood that the dynamic linker works in very different ways than the static linker. If I understood right, when linking a shared library, all symbols need to be resolved as the whole library is loaded in the memory.
A simple workaround would be to add all the dependencies of my libraries when compiling the executable. But they're so full of dependencies that this sometimes means adding 10+ libraries to the command line, and this would be for something like a hundred executable.
So far I tried using -Wl,--as-needed, -Wl,--unresolved-symbols=ignore-in-shared-libs, and opening the shared object with dlopen to get the function I want with dlsym. But all of these methods fail at one point or another.
My question is: Are you forced to resolve every undefined symbol of a dynamic library when linking it against an executable ?
Details of dynamic linking and the kinds of objects involved vary across environments and toolchains. On Linux, where you say you are, and on Solaris, and several other UNIX-y platforms, you are looking at ELF objects and semantics.
So far I tried using -Wl,--as-needed,
-Wl,--unresolved-symbols=ignore-in-shared-libs,
These both have their full effect at (static) link time. The first tells the linker that the libraries following it on the command line should be linked in only if they resolve at least one as-yet undefined symbol. The latter tells the linker to not worry about resolving symbols in shared libraries included in the link. That has nothing to do with the behavior of the dynamic linker when you run the program.
and opening the shared object with dlopen to get the function I want with dlsym.
dlopen instructs the dynamic linker to link in a shared object at runtime that was not specified in the binary as a required shared library. Its behavior at that point can be modulated by the flags passed to dlopen, but the options available are not more than can be specified at link time. There is little reason to use dlopen when you actually know at link time what libraries you need.
Are you forced to resolve every undefined symbol of a dynamic library
when linking it against an executable ?
Focusing on ELF and the GNU toolchain, no. -Wl,--unresolved-symbols=ignore-in-shared-libs serves precisely the purpose of avoiding that. But as you've discovered, that comes with caveats.
In the first place, in every shared object, every symbol referring to data needs to be resolved at runtime by the dynamic linker, no matter how you linked the various shared objects, including the main program. This is primarily an operational consideration -- the dynamic linker has no way to defer resolving symbols referring to objects because it has no good way to trap attempts to access them.
On the other hand, it is possible to defer resolution of symbols referring to functions until their first use. In fact, this is the GNU linker's default, but you can reaffirm this by passing -Wl,-z,lazy to gcc when linking. Note well, however, that this sets a property of the object being linked, so you should ensure that every shared object is built with that link option (but ordinarily they are because, again, that's the default).
Additionally, you should be aware that the dynamic linker's behavior can be influenced by environment variables. In particular, lazy binding will be disabled if the dynamic linker finds LD_BIND_NOW set to a nonempty string in the runtime environment.
A simple workaround would be to add all the dependencies of my
libraries when compiling the executable. But they're so full of
dependencies that this sometimes means adding 10+ libraries to the
command line, and this would be for something like a hundred
executable.
And what's the big deal with that, really? Surely you have a well-factored Makefile (or several) to help you, so it shouldn't be a big deal to ensure that all the libraries are linked. Right?
But you should also consider refactoring your libraries, especially if "interdependent" means there are loops in the dependency graph. Dynamic linking is different from static linking, as you've discovered, and the differences are sometimes more subtle than those you're presently struggling with. Although it is not a hard rule, I urge you to avoid creating situations where the shared objects used by one process contain among them multiple definitions of the same external symbol, especially if that symbol is actually used.
Update
The above discussion focuses on linking shared libraries to an executable, but there is another important consideration: how the libraries themselves are linked. Each ELF object, whether executable or shared library, carries its own list of needed shared libraries. The dynamic linker will recursively include all of these in the list of shared libraries to be loaded (immediately) at program startup, notwithstanding its behavior with respect to lazy binding of symbols referring to functions.
Therefore, if you want an executable not to require a given shared library X, then not only that executable itself but also every shared library it does rely upon must avoid expressing a dependency on X. If some of the shared libs require X when used in conjunction with other programs, then that puts the onus on you to link in all the needed libraries when building those programs (otherwise, you can arrange to link only direct dependencies). You can tell the GNU linker to build shared libraries this way by passing it the --allow-shlib-undefined flag.
Here is a complete proof of concept:
main.c
int mul(int, int);
int main(void) {
return mul(2, 3);
}
mul.c
int add(int, int);
int mul(int x, int y) {
return x * y;
}
int mul2(int x, int y) {
return add(x, y) * add(x, -y);
}
Makefile
CC = gcc
LD = gcc
CFLAGS = -g -O2 -fPIC -DPIC
LDFLAGS = -Wl,--unresolved-symbols=ignore-in-shared-libs
SHLIB_LDFLAGS = -shared -Wl,--allow-shlib-undefined
all: main
main: main.o libmul.so
$(LD) $(CFLAGS) $(LDFLAGS) -o $# $^
libmul.so: mul.o
$(LD) $(CFLAGS) $(SHLIB_LDFLAGS) -o $# $^
clean:
rm -f main main.o libmul.so mul.o
Demo
$ make
gcc -g -O2 -fPIC -DPIC -c -o main.o main.c
gcc -g -O2 -fPIC -DPIC -c -o mul.o mul.c
gcc -g -O2 -fPIC -DPIC -shared -Wl,--allow-shlib-undefined -o libmul.so mul.o
gcc -g -O2 -fPIC -DPIC -Wl,--unresolved-symbols=ignore-in-shared-libs -o main main.o libmul.so
$ LD_LIBRARY_PATH=$(pwd) ./main
$ echo $?
6
$
Note that the -zlazy linker option discussed in comments is omitted, as it's the default.
I'm trying to link a static library to a shared library via a command like
g++ -shared obj.o archive.a -o libLib.so
But at run time I keep getting unresolved system that should be link at compile time from archive.a. I've tried
g++ -shared obj.o -Wl,-Bstatic archive.a -Wl,-Bdynamic -o libLib.so
and
g++ -shared obj.o -Wl,-whole-archive archive.a -Wl,-no-whole-archive -o libLib.so
with no success. I feel like I'm missing something basic here...
You practically cannot (i.e. should never) do that. Shared libraries should be position independent code, but static libraries are not.
If you want to link libaa into libfoo.so build or get a shared (PIC) library libaa.so, not a static (non-PIC) library libaa.a
So a file foo1.cc going into a shared library libfoo.so should be compiled as
g++ -c -fPIC -Wall -O foo1.cc -o foo1.pic.o
and the library will be linked as
g++ -shared foo1.pic.o foo2.pic.o -o libfoo.so
You could link another shared library libsmiling.so into libfoo.so e.g. by appending -lsmiling to the above command. But you can't link libraries into static libraries, just copy their members.
But a file going into a static library libfoo.a is just compiled as
g++ -Wall -O foo1.cc -o foo1.o
so when you extract that foo1.o member from libfoo.a it is not PIC (and that is very inefficient)
In principle you could put a non-PIC object code in a shared library. In practice, you should never do that, because the amount of relocation is so large that it defeats the purpose of shared libraries. If you did that, text memory won't be sharable and the dynamic linker would have a lot of relocation work.
Code inside .so shared objects should be PIC to permit ld.so to mmap it at different address segments in different processes.
So you could find a sequence of commands to link a static library into a shared one, e.g. extract using ar x all the members of the libbar.a then link all these extracted bar*.o with foo*.pic.o but that would be a mistake. Don't link static non-PIC libraries or object files to PIC shared libraries.
For details (with a focus on Linux) read ld.so(8), ld(1), ELF wikipage, Levine's book: Linkers and loaders, Drepper's paper: How To Write Shared Libraries
PS. Some very few static libraries contain PIC code because they could be used to make shared libraries. For example my Debian provides the libc6-pic package giving notably the /usr/lib/x86_64-linux-gnu/libc_pic.a static PIC library -usable to build some variant of libc.so -e.g. if I wanted to put my own malloc in libc.so! - without recompiling every glibc source file.
I was trying to follow along with the C++ headers tutorial here, and as the tutorial says I have the files main.cpp, add.cpp, and add.h. The only is that up until now I haven't been using an IDE and compiling straight from the command line.
But I can't seem to figure out how I would compile add.h and and add.cpp into a library.
As of now, if give the command: g++ -o main main.cpp add.h add.cpp, it compiles just fine and gives me a main.exe. But how would I make it so the library (containing add.h and add.cpp) would be precompiled, and saved as a dll? Is this something that's relatively straight forward with the command line?
Thanks for any help guys, cheers.
There are two types of libraries: static and dynamic libraries. Static libraries are linked together with the resulting program, so each program that uses that library will get its own copy of the library code.
A more memory-efficient way is to use shared libraries (on windows called DLL), which are loaded on demand from a location that is specific for each platform, but the advantage is that only one instance of the library code needs to be loaded to memory when different programs that use the library are running simultaneously, and the resulting binary code of those programs do not contain the library code. it resides in a separate file that needs to be shipped together with the application and installed to a proper location.
If you use unix-like build tools (even on a windows system), this could be a typical sequence of commands you would use to produce a library that contains the code in your add.cpp file:
for a static library:
g++ -c add.cpp
ar crf libadd.a add.o
g++ -o main main.cpp -L. -ladd
the first will compile the add.cpp into add.o, the second will create a static library libadd.a from add.o file. If you want to include more object files into your library, add them to the end of that command line. The last command compiles your main.cpp program while linking it with the static library file libadd.a. The -L. option instructs the linker to search for the library file in the current directory. Alternately, you may want to put the library file in some other directory and use the -Lyour_directory option.
for a shared library (a dll):
g++ -shared -o libadd.so add.cpp
g++ -o main main.cpp -L. -ladd
but to run it, the system must be able to locate the shared library. You can help it by adding the directory where your library is located by adding it to the LD_LIBRARY_PATH environment variable, for instance:
export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
for the Windows platform, you may need to use a few more qualifiers, which are nicely explained in the mingw tutorial: http://www.mingw.org/wiki/sampledll
g++ -c main.cpp
g++ -c add.cpp
g++ - o x.dll main.o add.o
Suppose you're developing a shared library libshared.so.
And you have a static library libstatic.a with some internal classes and functionality you need. You'd like to link it to your .so like this:
g++ -o libshared.so -shared myObj.o -lstatic
Also you have an executable.sh which will use your .so and dynamically open it in the runtime
dlopen("libshared.so", RTLD_NOW)
You know this executable was as well statically linked against libstatic.a (but you're not sure the version of the library is exactly the same as yours).
So the question is:
Is it safe and correct to statically link your libshared.so against libstatic.a when you know the same library is already used in executable.sh?
You should avoid linking a static library into a shared one.
Because a shared library should have position independent code (otherwise, the dynamic linker has to do too much relocation, and you lose the benefits of shared libraries), but a static library usually does not have PIC.
Read Drepper's paper: How to write a shared library
You build your library with
g++ -Wall -O -fPIC mySrc.cc -c -o myObj.pic.o
g++ -o libshared.so -shared myObj.pic.o -lotherlib