Recently, I was trying to build an application, which uses some libraries, available in form of shared object files. I wasted lot of time in compiling the CPP code and it didn't work.
Below is the command, previously I was trying to compile the code-
g++ -I/opt/ros/indigo/include/ -I/usr/include/eigen3/ -L/opt/ros/indigo/lib/ -lorocos-kdl -lkdl_parser test.cpp -o test
The above command always shows many undefined references errors. Just for the curiosity, I changed the order of parameters. Below is the command, which is working-
g++ -L/opt/ros/indigo/lib -I/opt/ros/indigo/include -I/usr/include/eigen3 test.cpp -lorocos-kdl -lkdl_parser -o test
I posted the complete code and solution here.
My question is why does the order of passing parameters to g++ matter? Is there any alternative to avoid such problems in future?
Generally the order of arguments doesn't matter, but there are of course exceptions. For example if you provide multiple -O flags it will be the last one that is used, the same for other flags.
Libraries are a little different though, because for them the order is significant. If object file or library A depends on library B, then A must come before B on the command line. This is because of how the linker scans for symbols: When you use a library the linker will check if there are any symbols that could be resolved. Once this scan is over the library is discarded and will not be searched again.
This means when you have -lorocos-kdl -lkdl_parser test.cpp the linker will scan the libraries orocos-kdl and kdl_parser first, notice that there aren't dependencies on these library, no symbols from the libraries are needed, and continue with the object file generated by the source file.
When you change the order to test.cpp -lorocos-kdl -lkdl_parser the linker will be able to resolve the undefined symbols referenced by test.cpp when it comes to the libraries.
You can (at least in some versions of gcc) use parenthesis around the libraries if you don't want to care about the order.
See:
Why does the order in which libraries are linked sometimes cause errors in GCC?
Specifically:
If a static library depends on another library, but the other library
again depends on the former library, there is a cycle. You can resolve
this by enclosing the cycling dependent libraries by -( and -), such
as -( -la -lb -) (you may need to escape the parens, such as -( and
-)). The linker then searches those enclosed lib multiple times to ensure cycling dependencies are resolved.
Related
I'm trying to build my executable (that depends on library utils.so) using the following command
g++ -L/path/to/libutils -lutils -I/path/to/utils_headers executable.cpp -o executable
Actually I don't have utils.so - only the header files of utils library.
I'm getting the error:
ld: cannot find -lutils
Does linker really need to access all the libraries my executable depends on in order to build my executable? If it does then I'd like to know why it needs to access them.
My executable is a shared library. I'm sure that header files of the utils lib are enough to build it (i.e without having utils.so).
The linkage option -lutils by default directs the linker to search,
first in the specified library search directories (-Ldir) and then
in its default search directories, for either of the files libutils.so (
shared library) or libutils.a (static library), preferring libutils.so
if both of them are found in the same search directory.
If such a file is found, the linker stops searching and adds that file
to the input files of the linkage, whether or not it resolves any references in
the linkage. The linker cannot know whether the file resolves any references
if it does not input the file.
If no such file is found, the linker gives the error: cannot find -lutils. Because
you told it to find libutils.{so|a} and it could not.
You say:
My executable is a shared library
But it isn't. Your compile-and-link command:
$ g++ -L/path/to/libutils -lutils -I/path/to/utils_headers executable.cpp -o executable
is not an attempt to link a shared library. It is an attempt to link a program.1
This would be an attempt to link a shared library:
$ g++ -shared -I/path/to/utils_headers -o libexecutable.so executable.cpp -L/path/to/libutils -lutils
You cannot link a program with unresolved references. But you can link a shared library
with unresolved references.
So, you could link a libexecutable.so like that, or you could link it simply like:
$ g++ -shared -I/path/to/utils_headers -o libexecutable.so executable.cpp
These are two different linkages: if they succeed they produce different output files.
In the first linkage, some symbols will (let's assume) be resolved to definitions provided in libutils.so or libutils.a
(whichever one is found), and this will be reflected by:
libutils.so is found: The .dynamic section of libexecutable.so contains a DT_NEEDED
structure that expresses a runtime dependency on libutils.so. libutils.so will need to be included in any linkage that includes libexecutable.so, but the output file of such a linkage will itself contain a runtime dependency only on libexecutable.so.
libutils.a is found: libexecutable.so itself contains the definitions for all the symbols
it uses that are defined by object files in libutils.a.2 libexecutable.so may be included in subsequent linkages with no need for libutils.{so|a}.
In the second linkage, the .dynamic section of libexecutable.so will not express a runtime
dependency on libutils.so nor will the file contain definitions of any symbols provided by libutils.{so|a}. libutils.so will (again) need to be included in an subsequent linkage that includes libexecutable.so, but the output file of such a linkage will acquire independent runtime dependencies on both libexecutable.so and libutils.so.
But, if you specify -lutils in the linkage - or any linkage - and the linker cannot find libutils.{so|a}
in any of its search directories, then you get the error you observe, because you told the linker
to input a file, whose effects on the linkage can only be determined and implemented if that file is found - and it cannot be found.
[1] An attempt that is likely to fail, because it consumes libraries before the object
files that refer to them
[2] See static-libraries to understand
why.
In general, an ELF linker needs a sufficiently accurate representation of the shared object that is linked in. It does not have to be an actually working shared objects, just a sufficiently close representation of it. A few things absolute require data that is not available in the object itself:
When compiling C programs, a reference to a global data object of incomplete type does not contain size information. The linker cannot place the object into the data segment unless it obtains the size information from somewhere. By default (when compiling for executables, including PIE) the object needs to be allocated in the data segment on many targets because of the relocations the compiler uses for compiling accesses to global data objects.
Similarly, the link editor might get the alignment of global data objects wrong if it has insufficient information.
Many libraries use symbol versioning. Symbol version information is only available when the link editor can see the shared object. If that information is missing, the link editor will not emit a symbol version, which instructs the dynamic linker to bind the symbol to the base version at run time, leading to subtle bugs.
However, if you only use C function symbols (not data symbols, or the varieties of symbols that C++ requires) and the target library does not use symbol versioning, you can use a stub library for linking. This is a library that defines all the functions you need and has the appropriate soname, but the functions are just dummies which do not actually do anything.
I was recently reading this answer and noticed that it seems inconvenient for users to have to link static libraries in the correct order.
Is there some flag or #pragma I can pass to gcc when compiling my library so that my library's object files will always be included?
To be more specific, I want it to be case that the user can link my static library before another library that depends on it, and not wind up with unresolved symbols, in the same way that object files which are explicitly specified on the linker line are.
In particular, I am looking for solutions where the user of the library does not need to do anything special, and merely needs to pass -lMylibrary the way they would link any other library.
Is there some flag or #pragma I can pass to gcc
No.
I want it to be case that the user can link my static library before another library that depends on it, and not wind up with unresolved symbols, in the same way that object files which are explicitly specified on the linker line are.
Ship your "library" as a single object file. In other words, instead of:
ar ru libMyLibrary.a ${OBJS}
use:
ld -r -o libMyLibrary.a ${OBJS}
In particular, I am looking for solutions where the user of the library does not need to do anything special, and merely needs to pass -lMylibrary the way they would link any other library.
You can name your object file libMyLibrary.a. I believe the linker will search for it using the usual rules, but when it finds it, it will discover that this is an object file, and treat it as such, despite it being "misnamed". This should work at least on Linux and other ELF platforms. I am not sure whether it will work on Windows.
When compiling a program with static libraries, it was suggested to me from many sources (including SO community) to include the library twice.
As in:
gcc main.c -lslA -lslB -lslC -lslA -lslB -o final
Does this result in a bigger executable (.i.e. is the linker smart enough to avoid double inclusion?).
Is this (multiple inclusion) the proper solution or a workaround (.i.e. will there always exist a more proper, even if harder way to handle it)
The only reason to include the library multiple times is, for example, if slA requires a symbol resolved by slB but slB requires a symbol required by slA. The linker does a single pass to resolve symbols, but repeating your library causes, in effect, a second pass on that library. It won't change the size of your output, but it's not necessary either:
Instead of presenting your libraries multiple times, you can tell the gcc linker to group certain libraries together -- letting it do what it needs to resolve the symbols within that group. For example:
gcc main.c -Wl,--start-group -lslA -lslB -lslC -Wl,--end-group -o final
I'm attempting to design a shared library of shared libraries using g++ with hopes of simplifying my compile scripts and easing my update process in the future, but I'm still novice at best with GNU tools and writing libraries, at that. Can anyone provide advice on whether the following idea is possible with g++?
For convenience, consider the following file system layout:
main.cpp
libraryX/
libraryX/libX.so
libraryX/libraryY/
libraryX/libraryY/libY.so
libraryX/libraryZ/
libraryX/libraryZ/libZ.so
My goal is to be able to link indirectly using cascading relative paths. For instance, main.cpp links to libraryX/libX.so, which links to libraryY/libY.so and libraryZ/libZ.so. Is it possible to only link main.cpp to libX.so and use functions defined in libY.so and libZ.so?
If so, could you provide an example of the flags one would need to do so? I've been trying variations of the following command using various sources from Google to no avail:
g++ -shared -fPIC -Wl-rpath=libraryX -LlibraryX -lX.so main.o -o executable
Any guidance or references are greatly appreciated.
Don't do this (even if you can figure out how).
When you link against -lX, the static linker must know all other shared libraries that are "part of this link". Since -lY is not on the link line, the static linker will either give you an error, or it must somehow figure out where libY.so is coming from. For the latter, it has to replicate the RPATH search that the runtime loader will perform. This replication is error prone (the static linker may not use the exact same algorithm) and best avoided.
Finally, your command line is totally wrong: -shared means you ask the linker for a shared library, but you are clearly trying to link an executable. You generally should not use -fPIC when linking an executable. Also, -Wl-rpath=... should be -Wl,-rpath=... (the comma is important).
Prior to today I had always believed that the order that objects and libraries were passed to g++ during the linking stage was unimportant. Then, today, I tried to link from c++ code to c code. I wrapped all the C headers in an extern "C" block but the linker still had difficulties finding symbols which I knew were in the C object archives.
Perplexed, I created a relatively simple example to isolate the linking error but much to my surprise, the simpler example linked without any problems.
After a little trial and error, I found that by emulating the linking pattern used in the simple example, I could get the main code to link OK. The pattern was object code first, object archives second eg:
g++ -o serverCpp serverCpp.o algoC.o libcrypto.a
Can anyone shed some light on why this might be so?. I've never seen this problem when linking ordinary c++ code.
The order you specify object files and libraries is VERY important in GCC - if you haven't been bitten by this before you have lead a charmed life. The linker searches symbols in the order that they appear, so if you have a source file that contains a call to a library function, you need to put it before the library, or the linker won't know that it has to resolve it. Complex use of libraries can mean that you have to specify the library more than once, which is a royal pain to get right.
The library order pass to gcc/g++ does actually matter. If A depends on B, A must be listed first. The reason is that it optimizes out symbols that aren't referenced, so if it sees library B first, and no one has referenced it at that point then it won't link in anything from it at all.
A static library is a collection of object files grouped into an archive. When linking against it, the linker only chooses the objects it needs to resolve any currently undefined symbols. Since the objects are linked in order given on the command line, objects from the library will only be included if the library comes after all the objects that depend on it.
So the link order is very important; if you're going to use static libraries, then you need to be careful to keep track of dependencies, and don't introduce cyclic dependencies between libraries.
You can use --start-group archives --end-group
and write the 2 dependent libraries instead of archives:
gcc main.o -L. -Wl,--start-group -lobj_A -lobj_b -Wl,--end-group