I have a function in libA.so that is used in libB.so,
And a function in libB.so that is used in libA.so!
So defenetly i can not compile none of these libraries.
How can I compile these two libraries?
Should i used third library and move the dependebcies to this library?
I used qt and c++
Updated:
in compile libA.so get error cannot find libB.so and in libB.so get error can not find libA.so
BIG FAT DISCLAIMER Only do this if absolutely necessary. The preferred way is to refactor your project structure such that it doesn't contain dependency cycles.
When producing a shared library, the linker in general does not need to know about other shared libraries. One can use them on the command line but this is optional. Example:
// libA.cpp
extern void funcB();
void funcA() {
funcB();
}
Compile and link:
g++ -fPIC -c libA.cpp
g++ -shared -o libA.so libA.o
funcB is supposed to live in libB.so but we are not telling the linker where to find it. The symbol is simply left undefined in libA.so, and will be (hopefully) resolved at load time.
// libB.cpp
extern void funcA();
void funcB() {
funcA();
}
Compile and link, now using libA.so explicitly (ignore the infinite recursion, it's just an example):
g++ -fPIC -c libB.cpp
g++ -shared -o libB.so libB.o -L/where/libA/is -lA
Now it is up to the executable to load libB.so before loading libA.so, otherwise libA.so cannot be loaded. It's easy to do so (just link the executable with only libB.so and not libA.so), but can be inconvenient at times. So one can re-link libA.so after building libB.so:
g++ -shared -o libA.so libA.o -L /where/libB/is -lB
Now one can link an executable to libA or libB and the other one will be picked up automatically.
This seems a bit problematic for future re-use, you might want to either separate your functions differnetly between those libraries or create a third one thatt contains all of the "tool" funtions to have LibA and libB function without one another .
I have a function in libA.so that is used in libB.so, And a function in libB.so that is used in libA.so!
This is wrong design. A library cannot, even indirectly, depend upon itself. Such a circularity is the symptom of something very wrong, and you are misunderstanding what a software library is (it is more than a random collection of functions, or of object files; it has to be somehow a "software module" and it is related to modular programming and often defines and implements completely a collection of related abstract data types).
So throw both libA.so and libB.so away. And make a single libAB.so containing all the code that you have put in both libA.so and libB.so shared objects (and not genuine libraries).
The answer from n.m. gives a technical way to solve your problem, but at heart your design is wrong and you are abusing libraries (and you cannot call your libA or your libB a library, even if you built them as some shared object in ELF).
You could also design your code by adding some indirection with callbacks, or closures or function pointers held in some variable or data (and provide some way to set these callbacks, or initialize the closures or the function pointers at runtime). Since you use Qt, consider also defining appropriately your new Qt signals and slots (they are based on some callback machinery).
Read Program Library HowTo and Drepper's How to Write Shared libraries paper for more.
Finally I solve it.
As #n.m. said we dont need to link libA.so and libB.so in compile time, so I remove -lA and -lB when build them and i didnt get any error. And In app that want to use libA.so or libB.so I linked them with -lA or -lB. So this work correctly.
Related
Consider the following scenario:
Shared Library libA.so ,with no dependencies.
Shared Library libB.so, with libA.so as its dependency.
I want to compile a binary file that links with the libB.
Should I link the binary with libB only or with libA either?
Is there any way to link only with the direct dependencies, letting the resolution of unresolved symbols from the dependencies for runtime?
I'm worried about the fact that the library libB implementation may change in the future, introducing other dependencies (libC, libD, libE for instance). Am I going to have problems with that?
In other words:
libA files: a.cpp a.h
libB files: b.cpp b.h
main program files: main.cpp
Of course, b.cpp includes a.h and main.cpp includes b.h.
Compilation commands:
g++ -fPIC a.cpp -c
g++ -shared -o libA.so a.o
g++ -fPIC b.cpp -c -I.
g++ -shared -o libB.so b.o -L. -lA
Which of the bellow options should I use?
g++ main.cpp -o main -I. -L. -lB
or
g++ main.cpp -o main -I. -L. -lB -lA
I couldn't use the first option. The linker complains about the unresolved symbols from the library libA. But it sound a little strange to me.
Thanks very much.
-- Updated comments:
When I link the binary, the linker will try to resolve all symbols from the main and the libB. However, libB has undefined symbols from the libA. That's why the linker complains about that.
That's why I need to link with the libA too.
However I found a way to ignore unresolved symbols from shared libraries.
Looks like I should use the following command line to do that:
g++ main.cpp -o main -I. -L. -lB -Wl,-unresolved-symbols=ignore-in-shared-libs
Looks like it is still possible to use the -rpath option.
However I need to understand it a little better.
Does anyone knows any possible pitfalls when using the -Wl,-unresolved-symbols=ignore-in-shared-libs option?
-- Updated comments 2:
-rpath should not be used for this purpose. It is useful to force a library to be found in a given directory. The -unresolved-symbol approach looks much better.
Thanks again.
It looks like you are most of the way there already. Well done with your investigation. Let's see if I can help clear up the 'why' behind it.
Here's what the linker is doing. When you link your executable ('main' above) it has some symbols (functions and other things) that are unresolved. It will look down the list of libraries that follow, trying to resolve unresolved symbols. Along the way, it finds that some of the symbols are provided by libB.so, so it notes that they are now resolved by this library.
However, it also discovers that some of those symbols use other symbols that are not yet resolved in your executable, so it now needs to resolve those as well. Without linking against libA.so, your application would be incomplete. Once it links against libA.so, all symbols are resolved and linking is complete.
As you saw, the use of -unresolved-symbols-in-shared-libs, doesn't fix the problem. It just defers it so that those symbols are resolved at run time. That's what -rpath is for: to specify the libraries to be searched at run time. If those symbols can't be resolved then, your app will fail to start.
It's not an easy thing to figure out library dependencies because a symbol could be provided by more than one library and be satisfied by linking against any one of them.
There is another description of this process here: Why does the order in which libraries are linked sometimes cause errors in GCC?
For dynamic linking only with direct dependencies you can use -Wl,--as-needed with adding the libs after -Wl,--as-needed:
gcc main.c -o main -I. -L. -Wl,--as-needed -lB -lA
For checking the direct dependencies you should use readelf instead of ldd because ldd also shows the indirect dependencies.
$ readelf -d main | grep library
0x0000000000000001 (NEEDED) Shared library: [libB.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
ldd also shows the indirect dependencies:
$ LD_LIBRARY_PATH=. ldd ./main
linux-vdso.so.1 (0x00007fff13717000)
libB.so => ./libB.so (0x00007fb6738ed000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb6734ea000)
libA.so => ./libA.so (0x00007fb6732e8000)
/lib64/ld-linux-x86-64.so.2 (0x00007fb673af0000)
If you use cmake, you can add the following lines to include only direct dependencies:
set(CMAKE_EXE_LINKER_FLAGS "-Wl,--as-needed ${CMAKE_EXE_LINKER_FLAGS}")
set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--as-needed ${CMAKE_SHARED_LINKER_FLAGS}")
Another option is to use libtool
If you change the g++ call to libtool --mode=compile g++ to compile the source code and then libtool --mode=link g++ to create the application off of libB, then libA will be linked automatically.
This is an interesting post - I was banging my head with this as well, but I think you miss a point here..
The idea is as follows, right ?
main.cpp =(depends)=> libB.so =(depends)=> libA.so
Let's further consider that ..
In a.cpp (and only there) you define a class / variable, let's call it "symA"
In b.cpp (and only there) you define a class / variable, let's call it "symB".
symB uses symA
main.cpp uses symB
Now, libB.so and libA.so have been compiled as you described above. After that, your first option should work, i.e.:
g++ main.cpp -o main -I. -L. -lB
I guess that your problem originates from the fact that
in main.cpp you also refer to symA
Am I correct?
If you use a symbol in your code, then that symbol must be found in an .so file
The whole idea of inter-referencing shared libraries (i.e. creating APIs), is that the symbols in the deeper layers are hidden (think of peeling onions) and not used.
.. i.e. don't refer to symA in your main.cpp, but only to symB instead (and let symB to refer symA only).
I am creating a shared library from a class from an example I got here C++ Dynamic Shared Library on Linux. I would like to call another shared library from the shared library created and then use it in the main program. So I have the myclass.so library and I want to call another library say anotherclass.so from the myclass.so library and then use this myclass.so library in the main program. Any idea on how I can do this please.
There is more than one way in which multiple shared libraries may be added to
the linkage of a program, if you are building all the libraries, and the program,
yourself.
The elementary way is simply to explicitly add all of the libraries to the
the linkage of the program, and this is the usual way if you are building only the
program and linking libraries built by some other party.
If an object file foo.o in your linkage depends on a library libA.so, then
foo.o should precede libA.so in the linkage sequence. Likewise if libA.so
depends on libB.so then libA.so should precede libB.so. Here's an illustration.
We'll make a shared library libsquare.so from the files:
square.h
#ifndef SQUARE_H
#define SQUARE_H
double square(double d);
#endif
and
square.cpp
#include <square.h>
#include <cmath>
double square(double d)
{
return pow(d,2);
}
Notice that the function square calls pow, which is declared in the
Standard header <cmath> and defined in the math library, libm.
Compile the source file square.cpp to a position-independent object file
square.o:
$ g++ -Wall -fPIC -I. -c square.cpp
Then link square.o into a shared library libsquare.so:
$ g++ -shared -o libsquare.so square.o
Next we'll make another shared library libcube.so from these files:
cube.h
#ifndef CUBE_H
#define CUBE_H
double cube(double d);
#endif
and
cube.cpp
#include <cube.h>
#include <square.h>
double cube(double d)
{
return square(d) * d;
}
See that the function cube calls square, so libcube.so is going to
depend on libsquare.so. Build the library as before:
$ g++ -Wall -fPIC -I. -c cube.cpp
$ g++ -shared -o libcube.so cube.o
We haven't bothered to link libsquare with libcube, even though libcube
depends on libsquare, and even though we could have, since we're building libcube.
For that matter, we didn't bother to link libm with libsquare. By default the
linker will let us link a shared library containing undefined references, and it
is perfectly normal. It won't let us link a program with undefined references.
Finally let's make a program, using these libraries, from this file:
main.cpp
#include <cube.h>
#include <iostream>
int main()
{
std::cout << cube(3) << std::endl;
return 0;
}
First, compile that source file to main.o:
$ g++ -Wall -I. -c main.cpp
Then link main.o with all three required libraries, making sure to list
the linker inputs in dependency order: main.o, libcube.so, libsquare.so, libm.so:
$ g++ -o prog main.o -L. -lcube -lsquare -lm
libm is a system library so there's no need to tell the linker where to look for
it. But libcube and libsquare aren't, so we need to tell the linker to look for
them in the current directory (.), because that's where they are. -L. does that.
We've successfully linked ./prog, but:
$ ./prog
./prog: error while loading shared libraries: libcube.so: cannot open shared object file: No such file or directory
It doesn't run. That's because the runtime loader doesn't know where to find libcube.so (or libsquare.so, though it didn't get that far).
Normally, when we build shared libraries we then install them in one of the loader's default
search directories (the same ones as the linker's default search directories), where they're available to any program, so this wouldn't happen. But I'm not
going to install these toy libraries on my system, so as a workaround I'll prompt the loader where to look
for them by setting the LD_LIBRARY_PATH in my shell.
$ export LD_LIBRARY_PATH=.
$ ./prog
27
Good. 3 cubed = 27.
Another and better way to link a program with shared libraries that aren't located
in standard system library directories is to link the program using the linker's
-rpath=DIR option. This will write some information into the executable to tell
the loader that it should search for required shared libraries in DIR before it tries
the default places.
Let's relink ./prog that way (first deleting the LD_LIBRARY_PATH from the shell so that it's not effective any more):
$ unset LD_LIBRARY_PATH
$ g++ -o prog main.o -L. -lcube -lsquare -lm -Wl,-rpath=.
And rerun:
$ ./prog
27
To use -rpath with g++, prefix it with -Wl, because it's an option for linker, ld,
that the g++ frontend doesn't recognise: -Wl tells g++ just to pass the
option straight through to ld.
I would like to add some points to the response of #Mike.
As you do not link libcube library with libsquare you are creating a sort of "incomplete library". When I say incomplete, I meant that when you link your application you must link it with both libcube and libsquare even though it does not use any symbol directly from libsquare.
It is better to link libcube directly with libsquare. This link will create the library with a NEEDED entry like:
readelf -d libcube.so
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libsquare.so]
Then when you link your application you can do:
g++ -o prog main.o -L. -lcube
Although, this will not link because the linker tries to locate the NEEDED library libsquare. You must precise its path by adding -Wl,-rpath-link=. to the linking command:
g++ -o prog main.o -L. -lcube -Wl,-rpath-link=.
Note: For runtime, you must still set LD_LIBRARY_PATH or link with rpath as mentioned by #Mike.
In your library if you are using any other shared library so simply your library user is also dependent on that library. While creating library you can use -l so the linker have notion for shared library and it will link when required.
But when you deliver your library as its dependent on some other library you need to export that too along with your and provide some environment variable or linker flag to load it from specified path (Your exported package). That will not lead any discrepancy other wise if its some standard library function user might get definition from his system's some other library and will lead in disastrous situation.
Simply use the library like you'd use it in any other application. You don't have to link to anotherclass.so, just to myclass.so.
However, you will have to make both libraries (myclass.so and anotherclass.so) available for your later application's runtime. If one of them is missing you'll get runtime errors just like it is with any other application.
I'm working on a heavy project that has a lot of static libraries that are interdependent. Furthermore some symbols are redundant between some libraries, with different implementations. My goal is to make the project work with shared libraries.
I tried to compile an executable with one of my shared libs, and I get undefined symbols errors on functions that my executable isn't using. After some research I understood that the dynamic linker works in very different ways than the static linker. If I understood right, when linking a shared library, all symbols need to be resolved as the whole library is loaded in the memory.
A simple workaround would be to add all the dependencies of my libraries when compiling the executable. But they're so full of dependencies that this sometimes means adding 10+ libraries to the command line, and this would be for something like a hundred executable.
So far I tried using -Wl,--as-needed, -Wl,--unresolved-symbols=ignore-in-shared-libs, and opening the shared object with dlopen to get the function I want with dlsym. But all of these methods fail at one point or another.
My question is: Are you forced to resolve every undefined symbol of a dynamic library when linking it against an executable ?
Details of dynamic linking and the kinds of objects involved vary across environments and toolchains. On Linux, where you say you are, and on Solaris, and several other UNIX-y platforms, you are looking at ELF objects and semantics.
So far I tried using -Wl,--as-needed,
-Wl,--unresolved-symbols=ignore-in-shared-libs,
These both have their full effect at (static) link time. The first tells the linker that the libraries following it on the command line should be linked in only if they resolve at least one as-yet undefined symbol. The latter tells the linker to not worry about resolving symbols in shared libraries included in the link. That has nothing to do with the behavior of the dynamic linker when you run the program.
and opening the shared object with dlopen to get the function I want with dlsym.
dlopen instructs the dynamic linker to link in a shared object at runtime that was not specified in the binary as a required shared library. Its behavior at that point can be modulated by the flags passed to dlopen, but the options available are not more than can be specified at link time. There is little reason to use dlopen when you actually know at link time what libraries you need.
Are you forced to resolve every undefined symbol of a dynamic library
when linking it against an executable ?
Focusing on ELF and the GNU toolchain, no. -Wl,--unresolved-symbols=ignore-in-shared-libs serves precisely the purpose of avoiding that. But as you've discovered, that comes with caveats.
In the first place, in every shared object, every symbol referring to data needs to be resolved at runtime by the dynamic linker, no matter how you linked the various shared objects, including the main program. This is primarily an operational consideration -- the dynamic linker has no way to defer resolving symbols referring to objects because it has no good way to trap attempts to access them.
On the other hand, it is possible to defer resolution of symbols referring to functions until their first use. In fact, this is the GNU linker's default, but you can reaffirm this by passing -Wl,-z,lazy to gcc when linking. Note well, however, that this sets a property of the object being linked, so you should ensure that every shared object is built with that link option (but ordinarily they are because, again, that's the default).
Additionally, you should be aware that the dynamic linker's behavior can be influenced by environment variables. In particular, lazy binding will be disabled if the dynamic linker finds LD_BIND_NOW set to a nonempty string in the runtime environment.
A simple workaround would be to add all the dependencies of my
libraries when compiling the executable. But they're so full of
dependencies that this sometimes means adding 10+ libraries to the
command line, and this would be for something like a hundred
executable.
And what's the big deal with that, really? Surely you have a well-factored Makefile (or several) to help you, so it shouldn't be a big deal to ensure that all the libraries are linked. Right?
But you should also consider refactoring your libraries, especially if "interdependent" means there are loops in the dependency graph. Dynamic linking is different from static linking, as you've discovered, and the differences are sometimes more subtle than those you're presently struggling with. Although it is not a hard rule, I urge you to avoid creating situations where the shared objects used by one process contain among them multiple definitions of the same external symbol, especially if that symbol is actually used.
Update
The above discussion focuses on linking shared libraries to an executable, but there is another important consideration: how the libraries themselves are linked. Each ELF object, whether executable or shared library, carries its own list of needed shared libraries. The dynamic linker will recursively include all of these in the list of shared libraries to be loaded (immediately) at program startup, notwithstanding its behavior with respect to lazy binding of symbols referring to functions.
Therefore, if you want an executable not to require a given shared library X, then not only that executable itself but also every shared library it does rely upon must avoid expressing a dependency on X. If some of the shared libs require X when used in conjunction with other programs, then that puts the onus on you to link in all the needed libraries when building those programs (otherwise, you can arrange to link only direct dependencies). You can tell the GNU linker to build shared libraries this way by passing it the --allow-shlib-undefined flag.
Here is a complete proof of concept:
main.c
int mul(int, int);
int main(void) {
return mul(2, 3);
}
mul.c
int add(int, int);
int mul(int x, int y) {
return x * y;
}
int mul2(int x, int y) {
return add(x, y) * add(x, -y);
}
Makefile
CC = gcc
LD = gcc
CFLAGS = -g -O2 -fPIC -DPIC
LDFLAGS = -Wl,--unresolved-symbols=ignore-in-shared-libs
SHLIB_LDFLAGS = -shared -Wl,--allow-shlib-undefined
all: main
main: main.o libmul.so
$(LD) $(CFLAGS) $(LDFLAGS) -o $# $^
libmul.so: mul.o
$(LD) $(CFLAGS) $(SHLIB_LDFLAGS) -o $# $^
clean:
rm -f main main.o libmul.so mul.o
Demo
$ make
gcc -g -O2 -fPIC -DPIC -c -o main.o main.c
gcc -g -O2 -fPIC -DPIC -c -o mul.o mul.c
gcc -g -O2 -fPIC -DPIC -shared -Wl,--allow-shlib-undefined -o libmul.so mul.o
gcc -g -O2 -fPIC -DPIC -Wl,--unresolved-symbols=ignore-in-shared-libs -o main main.o libmul.so
$ LD_LIBRARY_PATH=$(pwd) ./main
$ echo $?
6
$
Note that the -zlazy linker option discussed in comments is omitted, as it's the default.
For background, I'm creating some C++ software that uses dynamically loaded shared library plugins for hardware output (the specifics of it aren't relevant here).
I'm building the executable by compiling everything into object files and then linking the ones needed, which is simple using an exclusion list. I can then build the shared library by specifying its primary object file (the one that's dynamically loaded and accessed at runtime) along with every other object file referenced by the primary one.
My question is this: Is there a way to provide the linker with the primary object file, and create a shared library containing only the objects it depends upon? All of the object files are in the same directory, I'm not using a Makefile (yet; if one could solve the problem, it's a valid answer), and compilation speed isn't an issue.
I've looked into the linker options --as-needed, --gc-sections, and --no-undefined, but I haven't been able to piece together a working build process.
Example: For source files main.cpp, a.cpp, b.cpp, a.h, and b.h, where main.cpp and a.cpp both include b.h:
gcc -fPIC -c *.cpp -I. builds object files main.o, a.o, and b.o.
gcc -o main.out *.o builds the final executable main.out from the object files... including a.o, which is unused. (--gc-sections should fix this.)
gcc -fPIC -shared -o a.so a.o -Wl,--as-needed !(a).o builds the final shared library a.so from all of the object files... including main.o, which is unused. How do I prevent main.o from being included in a.so?
Is there a way to provide the linker with the primary object file, and create a shared library containing only the objects it depends upon?
Yes: package all objects into an archive library liball.a, then link like this:
gcc -shared -o a.so a.o liball.a
The linker will then pull out from liball.a all objects that a.o depends on, and only these objects, as explained here.
Note: liball.a may contain a.o, there is no harm (as above link explains).
Update:
Is there a way to do it without needing to create an archive first?
I don't know of any portable way to do that. The Gold linker has --start-lib and --end-lib command line flags that achieve exactly that.
Suppose you're developing a shared library libshared.so.
And you have a static library libstatic.a with some internal classes and functionality you need. You'd like to link it to your .so like this:
g++ -o libshared.so -shared myObj.o -lstatic
Also you have an executable.sh which will use your .so and dynamically open it in the runtime
dlopen("libshared.so", RTLD_NOW)
You know this executable was as well statically linked against libstatic.a (but you're not sure the version of the library is exactly the same as yours).
So the question is:
Is it safe and correct to statically link your libshared.so against libstatic.a when you know the same library is already used in executable.sh?
You should avoid linking a static library into a shared one.
Because a shared library should have position independent code (otherwise, the dynamic linker has to do too much relocation, and you lose the benefits of shared libraries), but a static library usually does not have PIC.
Read Drepper's paper: How to write a shared library
You build your library with
g++ -Wall -O -fPIC mySrc.cc -c -o myObj.pic.o
g++ -o libshared.so -shared myObj.pic.o -lotherlib