Linker seems to ignore library and default to older version - c++

I've built a debug version of a shared library (OpenSSL) so I can step through a certain function with a debugger to better understand what's going on.
However, I'm having a difficult time actually linking with the debug version I've built. For some reason, no matter what I do, the linker always ends up linking with the pre-installed system version, even though both versions are in the usr/lib directory, the soft-links are setup correctly (AFAIK), and I explicitly specify the debug lib on the command line when compiling.
So the original (system-installed) version of the shared library is:
>ls /usr/lib/x86_64-linux-gnu/ -lh | grep libssl
lrwxrwxrwx 1 root root 15 Sep 23 2016 libssl.so -> libssl.so.1.0.0
-rw-r--r-- 1 root root 386K Sep 23 2016 libssl.so.1.0.0
And the debug version, which I compiled from source and configured as a shared library (using the fPIC flag for all object files), is:
>ls /usr/lib/ -lh | grep libssl
lrwxrwxrwx 1 root root 29 Oct 19 11:31 libssldebug.so -> /usr/lib/libssldebug.so.1.0.2
-rwxr-xr-x 1 root root 2.3M Oct 19 00:53 libssldebug.so.1.0.2
And it's the same with the other OpenSSL shared library, libcrypto. I have a libcryptodebug.so.1.0.2 and a corresponding soft link in /usr/lib.
So, I try to build an executable and link against the debug shared lib like this:
>g++ test.cpp -o test -std=c++14 -lssldebug -lcryptodebug -I openssl-1.0.2p/include/
And it compiles and links with no errors.
And YET... when I examine the executable with ldd, I see:
>ldd test
linux-vdso.so.1 (0x00007ffcaa39b000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007ff717d37000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007ff71793b000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ff717630000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff71732f000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff717119000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff716d6e000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff716b6a000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff717f98000)
So even though I explicitly linked with -lssldebug, and it compiled and linked with no errors, ldd still shows that the linker for some reason is linking with the non-debug version (/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0). I also tried running ldconfig and then recompiling/linking, but it still links with the old (non-debug) version.
So what is going on here? What am I doing incorrectly that causes it to silently ignore the command line linker arguments and somehow just default to using the non-debug version (which has a completely different shared lib name!) somehow?

So even though I explicitly linked with -lssldebug, and it compiled and linked with no errors, ldd still shows that the linker for some reason is linking with the non-debug version (/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
You are mixing up static linking, and runtime loading (sometimes also called dynamic linking).
When you link with g++ test.cpp ... -lssldebug ..., you are linking with libssldebug.so, but (as ldd output tells you) that library is not used during runtime loading.
That is happening because libssldebug.so has a special dynamic tag DT_SONAME, which contains "libssl.so.1.0.0", and the static linker records that name as the library to be loaded at runtime.
You can confirm this with:
readelf -d libssldebug.so | grep SONAME
You can examine the libraries that runtime loader will try to locate for a given executable with:
readelf -d ./test | grep NEEDED
Now that you understand the problem, how do you fix it?
One of two ways:
You could change the SONAME encoded in libssldebug.so, by relinking it with -Wl,--soname=libssldebug.so.1.0.2, and then relinking your test program. Use above readelf commands to verify that SONAME and NEEDED now contain "libssldebug.so.1.0.2", and then ldd to verify that libssldebug.so.1.0.2 is what the runtime loader will use.
Alternatively, it may be simpler to install libssldebug.so into a different directory (say /tmp/libssldebug), create a symlink libssl.so.1.0.0 -> lissldebug.so in that directory, and then ask the runtime loader to search this directory first with g++ test.cpp ... -Wl,--rpath=/tmp/libssldebug. With this solution, the executable will still search for libssl.so.1.0.0, but it will search for it in the /tmp/libssldebug directory first, and will find your copy.

Related

Why linux ld somtime use the symbolic link of libs

I use the jsoncpp lib in my linux cli tool.
The CMakeLists.txt contains
find_library(LIB_JSON jsoncpp)
target_link_libraries(${PROJECT_NAME} ${LIB_JSON})
The result is
/usr/bin/c++ -rdynamic CMakeFiles/cktwagent.dir/agent_main.cpp.o -o cktwagent -ljsoncpp
When i check the binary I found:
$> ldd cktwagent
linux-vdso.so.1 (0x00007ffe4cfd1000)
libjsoncpp.so.24 => /usr/lib/libjsoncpp.so.24 (0x00007f87505bd000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f87503e0000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f875029a000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f8750280000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f87500b7000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f87506ce000)
Why ld use /usr/lib/libjsoncpp.so.24 and not the symbolic link /usr/lib/libjsoncpp.so?
Why the ld sometime resolv the linbrary link to the real library file?
$> ls -l /usr/lib/libjsoncpp.so
lrwxrwxrwx 1 root root 16 26. Sep 17:02 /usr/lib/libjsoncpp.so -> libjsoncpp.so.24
In case of /usr/lib/libstdc++.so.6 the ld use the symbolic link. When i check the path from ldd output, libstdc++.so.6 point to a symbolic link.
$> ls -l /usr/lib/libstdc++.so.6
lrwxrwxrwx 1 root root 19 9. Nov 12:43 /usr/lib/libstdc++.so.6 -> libstdc++.so.6.0.28
I like to understand this behavior. Because when i copy the binary to a different system, the link to libjsoncpp.so available. But it points to some different version.
Many thanks
Thomas
That's a version compatibility feature. By convention, API compatibility is determined by the major version number. So 6.0.28 would be is compatible with 6.1.1, but not with 5.0.1 or 7.0.1.
To allow for compatible upgrades, libstdc++.so.6.0.28 is symlinked as libstdc++.so.6. Then it can be binary-upgraded to e.g. libstdc++.so.6.0.29 without touching applications that rely on API version 6 of libstdc++.
In addition, it allows libraries with different major version numbers to coexist on the same system. Installing libstdc++.so.7.0.0 will not break apps that link against libstdc++.so.6.
Why ld use /usr/lib/libjsoncpp.so.24 and not the symbolic link /usr/lib/libjsoncpp.so?
That's because libjsoncpp has a soname of libjsoncpp.so.24. Apparently it is following the same convention of having the major version number determine compatibility.
You can achieve the same behavior with your shared lib libfoo if you link it like this:
gcc -shared -Wl,-soname,libfoo.so.<major> -o libfoo.so.<major>.<minor>
For more details refer to GCC Howto - Version numbering, sonames and symlinks.

CMake project fails to find shared library

I am using CMake to build a cross platform project. For the moment I am trying to run it on Linux. I have recently added a project for running tests, but it will not run because it cannot find one of the shared libraries, specifically libtbbmalloc.so.2:
/tests: error while loading shared libraries: libtbbmalloc.so.2: cannot open shared object file: No such file or directory`
When I run ldd on the executable I get the following:
linux-vdso.so.1 (0x00007fffeb572000)
libtbbmalloc_proxy.so.2 => /home/username/dev/tbb/libtbbmalloc_proxy.so.2 (0x00007f50afe00000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f50afa70000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f50af6d0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f50af4b0000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f50af0a0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f50aee90000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f50aec70000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f50aea60000)
/lib64/ld-linux-x86-64.so.2 (0x00007f50b0400000)
libtbbmalloc.so.2 => not found
The CMakeLists.txt for my test project looks like this:
set(test_sourcefiles main_tests.cpp)
add_executable(tests ${test_sourcefiles})
target_link_libraries(tests Catch2::Catch2 MyLib)
MyLib uses tbb, and I guess that is why my executable (tests) searches for it. When running ldd on MyLib it finds the library (libtbbmalloc.so.2):
(removed some output for readability)
libtbbmalloc_proxy.so.2 => /home/username/dev/tbb/libtbbmalloc_proxy.so.2 (0x00007f9af8110000)
libtbbmalloc.so.2 => /home/username/dev/tbb/libtbbmalloc.so.2 (0x00007f9ac4eb0000)
I have tried specifically adding libttbbmalloc.so.2 in my tests/CMakeLists.txt target_link_libraries(${project} /home/username/dev/tbb/libtbbmalloc.so.2), but it makes no difference.
If I add /home/username/dev/tbb/ to LD_LIBRARY_PATH, the program runs, and ldd reports that libtbbmalloc.so.2 is found.
Any ideas on what I might be doing wrong, and how can I get my program to run without setting LD_LIBRARY_PATH?
Update: I found out that it's possible to print runpath/rpath by using chrpath -l name-of-executable. When using this tool on my executable, it looks like the folder with libtbbmalloc.so.2 is added to runpath, but the program still won't run:
larjr#DESKTOP:~/dev/project/build/tests$ chrpath -l tests
tests: RUNPATH=/home/larsjr/dev/project/build/MyLib:/home/username/dev/tbb
The issue you are encountering does sound to be related to the search path for direct and indirect linkage of shared libraries. CMake may build with the new ld RUNPATH variable which will only work on direct dependents, where RPATH will work recursively on all indirect dependents. As shown in the snippet at bottom of this post, this may depend on your version of GCC. To check if your executable is using RPATH or RUNPATH, run:
$ readelf -d ./executable
and find the library rpath / runpath line after your dependencies. ie/eg: It may show either of the below:
a) 0x000000000000001d (RUNPATH) Library runpath: [/some/dir/lib]
b) 0x000000000000001d (RPATH) Library rpath: [/some/dir/lib]
You showed you used the command chrpath -l ./executable, which actually spat out:
larjr#DESKTOP:~/dev/project/build/tests$ chrpath -l tests
tests: RUNPATH=/home/larsjr/dev/project/build/MyLib:/home/username/dev/tbb
It says here "RUNPATH" which is non-recursive - that's your error.
Refer to this excellent github issue where members are discussing indirect linking issues with CMake, highly suggest reading it all. It has a a great example regarding dependencies and their search paths.
https://github.com/conan-io/conan/issues/2660
A snippet regarding new vs old linker behavior which can cause this 'issue':
The problem you are experiencing in conan-deptest (excellent isolated case, by the way!) is the behaviour that you are documenting, between DT_RPATH and DT_RUNPATH.
It is CMake (and not conan), that passes the -rpath flag with the correct paths to the dependent libraries, when building the executable.
The problem is that newer versions of the linker in some platforms, emit DT_RUNPATH where they used to emit DT_RPATH. Indeed this is the case as readelf -d is listing the RUNPATH tag.
There is also another stackoverflow post where a user asks about this change in behaviour in LD/GCC:
How to set RPATH and RUNPATH with GCC/LD?
A conan member further elaborates on how to change this behavior:
The way to replicate this behaviour on versions of the linker that emit RUNPATH instead, is to pass the --disable-new-dtags flag to the linker.
And the resulting cmake:
target_link_libraries(target "-Wl,--disable-new-dtags")
Now if you rebuild, and check readelf -d on your executable, you should see the RPATH tag. You can run:
env LD_DEBUG=files,libs ./executable
to see RPATH being passed into the search path for each dependency:
31658: file=libbar.so.1 [0]; needed by /home/user/projectA/lib/libfoo.so.2.5 [0]
31658: find library=libbar.so.1 [0]; searching
31658: search path=/home/user/projectA/lib (RPATH from file ./executable)
31658: trying file=/home/user/projectA/lib/libbar.so.1
Hopefully this fixes your issue!
Bonus:
Link to tutorial on shared libraries. At the bottom of the aritcle shows runpath vs rpath + origin!
https://amir.rachum.com/blog/2016/09/17/shared-libraries/
I believe you are looking for setting a runtime path on the executable:
set_target_properties(tests PROPERTIES
INSTALL_RPATH "<your path to libs>"
BUILD_WITH_INSTALL_RPATH 1
)
INSTALL_RPATH
Normally CMake uses the build tree for the RPATH when building executables etc on systems that use RPATH. When the software is installed the executables etc are relinked by CMake to have the install RPATH. If this variable is set to true then the software is always built with the install path for the RPATH and does not need to be relinked when installed.
BUILD_WITH_INSTALL_RPATH
Normally CMake uses the build tree for the RPATH when building executables etc on systems that use RPATH. When the software is installed the executables etc are relinked by CMake to have the install RPATH. If this variable is set to true then the software is always built with the install path for the RPATH and does not need to be relinked when installed.
add set_target_properties next to your executable install command
# install
INSTALL(TARGETS ${PROJECT_NAME} myLib
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
)
set_target_properties(${PROJECT_NAME} PROPERTIES
INSTALL_RPATH ${CMAKE_INSTALL_PREFIX}/lib
BUILD_WITH_INSTALL_RPATH ON
)

Shipping libstdc++.so.6 with application

I want to use gcc 4.8.1 for my application (requires libstdc++.so.6.0.18), however customers only have libstdc++.so.6.0.13. I have been using -static-libgcc -static-stdlibc++ for a while now, but my application consists of several dynamically linked libraries and one main application. This means that when compiling each dynamic library, they must statically compile the standard library, which is redundant and wasteful. I want to just ship the standard library of my choice with my product, however every time I run my application in an environment like theirs, it always loads the wrong standard library. It prefers the /usr/lib64/ version no matter what I seem to do (it seems to take precedence over LD_LIBRARY_PATH).
Constraints:
I'm not allowed to force them to upgrade to a new standard library.
I don't want to make the dynamic libraries static. (I'd be able to statically compile everything into the main app once, but there are some logistical barriers that prevent me from recompiling some libraries into static ones).
-Wl,-rpath=$(path_to_directory) is a bit dangerous, however it is legal because the customers do source some settings that allow me to set path variables. However, setting the rpath of my new stdlibc++ doesn't seem to be overriding the default /usr/lib64 version. I still get GLIBCXX errors because it won't use the right library.
Surely there is an elegant solution for this?
Perhaps there is just an error in my procedure. Here's an example (sorry about the censor, but it's just username stuff):
~/example$ pwd
/home/username/example
~/example$ echo $LD_LIBRARY_PATH
~/example$ ls
Makefile libstdc++.so.6.0.18 test.cpp
~/example$ make
g++ -std=c++11 -Wall -Werror test.cpp -o test
~/example$ ldd test
./test: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by ./test)
linux-vdso.so.1 => (0x00007fffe5919000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000390b800000)
libm.so.6 => /lib64/libm.so.6 (0x0000003904800000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x000000390b400000)
libc.so.6 => /lib64/libc.so.6 (0x0000003904400000)
/lib64/ld-linux-x86-64.so.2 (0x0000003904000000)
~/example$ setenv LD_LIBRARY_PATH /home/username/example
~/example$ echo $LD_LIBRARY_PATH
/home/username/example
~/example$ ldd test
./test: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by ./test)
linux-vdso.so.1 => (0x00007fff2d3ff000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000390b800000)
libm.so.6 => /lib64/libm.so.6 (0x0000003904800000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x000000390b400000)
libc.so.6 => /lib64/libc.so.6 (0x0000003904400000)
/lib64/ld-linux-x86-64.so.2 (0x0000003904000000)
Sorry guys, I made a rather dumb mistake...
~/example$ file libstdc++.so.6.0.18
libstdc++.so.6.0.18: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped
Some dweeb built the wrong version of the library, and another dweeb (namely myself) tried using it on a 64-bit machine. Using LD_LIBRARY_PATH was working all along...
Your problem is that the executable is linked to the soname libstdc++.so.6 not to the full library filename libstdc++.so.6.0.16. The dynamic linker will look for libstdc++.so.6 in the usual places (i.e. LD_LIBRARY_PATH, DT_RPATH, ldconfig dirs etc.) so to ensure the 6.0.18 version is found you need a symlink called libstdc++.so.6 pointing to it.
Instead of using LD_LIBRARY_PATH (which is fragile on machines you don't control, because users might alter their environment) I prefer linking with '-Wl,-rpath,$ORIGIN' (N.B. the quotes are necessary to stop the shell expanding $ORIGIN)
An RPATH of $ORIGIN tells the dynamic linker to start looking for shared libraries in the same directory as the executable, so if you ship libstdc++.so alongside your executable it will be found. If you want to ship the executable in a bin directory and have the library in a lib directory you can use '-Wl,-rpath,$ORIGIN/../lib' or other paths relative to the location of the executable.

Compiling with -static-libgcc -static-libstdc++ still results in dynamic dependency on libc.so

I'm trying to make an executable that's as portable as possible. After removing a few dependencies, I came across the following when running the binary on another system:
/lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.15' not found (required by foob)
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.15' not found (required by foob)
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found (required by foob)
I'd prefer my binary not to require the user to upgrade their version of libc, so I'd like to remove this dependency as well.
The linker flags that produced the above binary already included -static-libgcc -static-libstdc++. How come the binary still requires on the shared libc.so.6?
I tried adding the -static flag as well, however when I try to run that binary the result is very strange:
$ ls -l foob
-rwxr-xr-x 1 claudiu claudiu 13278191 Oct 10 13:03 foob
$ ./foob
bash: ./foob: No such file or directory
What to do?
EDIT:
$ file foob
foob: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=5adee9a598b9261a29f1c7b0ffdadcfc72197cd7, not stripped
$ strace -f ./foob
execve("./foob", ["./foob"], [/* 64 vars */]) = -1 ENOENT (No such file or directory)
write(2, "strace: exec: No such file or di"..., 40strace: exec: No such file or directory
) = 40
exit_group(1) = ?
+++ exited with 1 +++
Interestingly, if I ldd the version without -static, it has two less entries than the version with -static, namely:
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f4f420c1000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4f41636000)
GNU libc is not designed to be statically linked. Important functions, e.g. gethostbyname and iconv, will malfunction or not work at all in a static binary. Arguably even worse, under some conditions a static binary will attempt to dynamically open and use libc.so.6, even though the whole point of static linkage is to avoid such dependencies.
You should compile your program against uClibc or musl libc instead.
(This has been true for at least 15 years.)
First be aware that static linking of libc might not improve portability of your program, as libc might depend on other parts of your system e.g. kernel version.
If you want to try complete static linking just using -static should the trick. Provided that there are static versions of all used libraries installed.
You can check if your program has only linked static libraries by using:
ldd binary_name
EDIT:
Another way that provides useful information for debugging this problem would be to add --verbose to your linker flags.

How to list exported functions in a shared lib on Ubuntu

I have just built a shared lib on Ubuntu, and when I attempt to use the function, the application that loads the library is reporting 'xxx' symbol not found.
I want to check (i.e. list) the functions that are exported by my library so I can investigate this issue further.
Relevant details:
OS: Ubuntu 9.10
compiler: gcc 4.4.1
linker: GNU ld 2.20
Try the nm utility.
GNU nm lists the symbols from object
files objfile.... If no object files
are listed as arguments, nm assumes
the file a.out. [reference]
nm -D -C -g <library>
works well too.
Is your shared library in the library load path or in the application's run-time search path? It sounds like the dynamic linker can't find your library. Try running ldd on your application to see if the library can be found at run-time, e.g.:
$ ldd /usr/bin/less
linux-gate.so.1 => (0x0072a000)
libncurses.so.5 => /lib/libncurses.so.5 (0x00c68000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x007c7000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0x00286000)
/lib/ld-linux.so.2 (0x002a1000)
See the ld.so(8) man page for additional details on library search paths.