Executable missing full library path after build - c++

I am using CMake to compile an executable that is linked against several libraries that I have built and installed into a local project directory (libs/3rdparty). Note that this is prior to installation of the project, primarily for the purpose of running unit tests and debugging. The problem I am having is that sometimes there is a library that is linked, but the executable is missing the path to the library. The library I am currently having an issue with is leptonica. However, I have run into this issue several times with different libraries on different platforms (osx, fedora, centos, ubuntu). Through research I have seen similar issues, but I have never been able to find a definitive answer of why the full path to the library would be missing.
I've tried playing with:
CMAKE_BUILD_WITH_INSTALL_RPATH
CMAKE_INSTALL_RPATH
CMAKE_INSTALL_RPATH_USE_LINK_PATH
and these don't seem to have much effect.
My CMakeLists contains:
find_package(Leptonica REQUIRED)
target_link_libraries(${target}
PRIVATE
...
${Leptonica_LIBRARIES}
)
Here is the output from ldd on one of the unit test executables:
ldd test_utilities
...
libleptonica.so.5.3.0 => not found
libtesseract.so.4 => {MY PROJECT}/libs/3rdparty/tesseract/lib/libtesseract.so.4
leptonica is the only library that is not found out of ~30 other libraries.
Does anyone know what the root cause of this problem is? I am not looking to work around the problem by modifying LD_LIBRARY_PATH.
-- Added LeptonicaTargets-release.cmake. According to this the full path to the lib should be in the target.
#----------------------------------------------------------------
# Generated CMake target import file for configuration "RELEASE".
#----------------------------------------------------------------
# Commands may need to know the format version.
set(CMAKE_IMPORT_FILE_VERSION 1)
# Import target "leptonica" for configuration "RELEASE"
set_property(TARGET leptonica APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE)
set_target_properties(leptonica PROPERTIES
IMPORTED_LINK_INTERFACE_LIBRARIES_RELEASE "/usr/lib/x86_64-linux-gnu/libpng.so;/usr/lib/x86_64-linux-gnu/libz.so;m"
IMPORTED_LOCATION_RELEASE "${_IMPORT_PREFIX}/lib/libleptonica.so.1.77.0"
IMPORTED_SONAME_RELEASE "libleptonica.so.5.3.0"
)
list(APPEND _IMPORT_CHECK_TARGETS leptonica )
list(APPEND _IMPORT_CHECK_FILES_FOR_leptonica "${_IMPORT_PREFIX}/lib/libleptonica.so.1.77.0" )
# Commands beyond this point should not need to know the version.
set(CMAKE_IMPORT_FILE_VERSION)
Here are the files in the leptonica/lib directory:
ll libs/3rdparty/leptonica/lib/
total 2776
drwxr-xr-x 3 user user 4096 May 30 14:17 ./
drwxr-xr-x 5 user user 4096 May 30 14:17 ../
lrwxrwxrwx 1 user user 21 May 30 14:17 libleptonica.so -> libleptonica.so.5.3.0
-rw-r--r-- 1 user user 2829784 May 30 09:49 libleptonica.so.1.77.0
lrwxrwxrwx 1 user user 22 May 30 14:17 libleptonica.so.5.3.0 -> libleptonica.so.1.77.0
drwxr-xr-x 2 user user 4096 May 30 14:17 pkgconfig/
Output from chrpath --list test_utilities appears to contain the correct path to the library as well:
chrpath --list test_utilities
test_utilities: RUNPATH=...:{MY PROJECT}/libs/3rdparty/leptonica/lib:...

For anyone who runs across this, I have finally figured it out.
The issue was related to the library being a transitive dependency of OpenCV. On Ubuntu, ld now defaults to using using --enable-new-dtags which uses RUNPATH, not RPATH. There is an issue where RUNPATH is not searched for transitive dependencies.
See https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/1253638
Simply adding "-Wl,--disable-new-dtags" to the target linker options resolved my issue. All libraries are now found, including other libraries than leptonica that I added today. I am sure that I will likely have to make changes when building a package for installation though.

Related

error while loading shared libraries: librocksdb.so.6.12: cannot open shared object file: No such file or directory

I am at my wits' end with this issue because I have no idea what is wrong with my compiler. I am trying to run some C++ code using cmake on Ubuntu 21, and I get this error:
error while loading shared libraries: librocksdb.so.6.12: cannot open shared object file: No such file or directory
Now here's the thing: I have rocksdb installed (via apt), but only the following:
ls -l /lib/librocksdb*
-rw-r--r-- 1 root root 21315638 Dec 10 2020 /lib/librocksdb.a
lrwxrwxrwx 1 root root 20 Dec 10 2020 /lib/librocksdb.so -> librocksdb.so.6.11.4
lrwxrwxrwx 1 root root 20 Dec 10 2020 /lib/librocksdb.so.6 -> librocksdb.so.6.11.4
I know the path is the correct one and is recognised by ld since it works with my other shared libraries. There is clearly a version mismatch since my compiler is trying to find 6.12, but I have no idea how to fix it and get the correct version. Things I tried so far:
clearing caches
reinstalling
building from source (does not generate these library names, even with the correct branch)
creating symlinks (still gives me the same error)
setting find_library(rocksdb NAMES librocksdb.so.6 REQUIRED) in my CmakeLists.txt (also tried with 6.11)
What am I doing wrong and why am I getting this error? Any help would be appreciated.
Fixed, I had some old header files under my /include directory and the wrong library was being linked at compile time

Custom-built and default boost libraries on the same system

I've built some boost libraries (particularly system and serialization) with the _GLIBCXX_USE_CXX11_ABI=0 macro (https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html) due to the fact that I'm using boost::asio to share information (by mean) between my code and an old application (Webots 7.4.3, pre GCC-5). However, I already have standard boost libraries installed in my system (required by several other applications). I've opted to install the custom-compiled versions to /usr/local/lib (while the default versions are kept in /usr/lib) and renamed the libraries with the suffix _oldabi:
$ ls -lash /usr/local/lib | grep boost
1,1M -rw-r--r-- 1 root root 1,1M jun 21 14:17 libboost_serialization_oldabi.a
416K -rw-r--r-- 2 root root 415K jun 21 14:17 libboost_serialization_oldabi.so
416K -rw-r--r-- 2 root root 415K jun 21 14:17 libboost_serialization_oldabi.so.1.64.0
0 lrwxrwxrwx 1 root root 39 jun 21 14:18 libboost_serialization.so.1.64.0 -> libboost_serialization_oldabi.so.1.64.0
48K -rw-r--r-- 1 root root 46K jun 21 14:17 libboost_system_oldabi.a
20K -rw-r--r-- 2 root root 20K jun 21 14:17 libboost_system_oldabi.so
20K -rw-r--r-- 2 root root 20K jun 21 14:17 libboost_system_oldabi.so.1.64.0
0 lrwxrwxrwx 1 root root 32 jun 21 14:18 libboost_system.so.1.64.0 -> libboost_system_oldabi.so.1.64.0
708K -rw-r--r-- 1 root root 706K jun 21 14:17 libboost_wserialization_oldabi.a
300K -rw-r--r-- 2 root root 299K jun 21 14:17 libboost_wserialization_oldabi.so
300K -rw-r--r-- 2 root root 299K jun 21 14:17 libboost_wserialization_oldabi.so.1.64.0
0 lrwxrwxrwx 1 root root 40 jun 21 14:18 libboost_wserialization.so.1.64.0 -> libboost_wserialization_oldabi.so.1.64.0
The code works well, socket communications work as intended. However, when trying to set up the code on another computer without the default boost libraries installed (only custom-compiled versions of boost::system and boost::serialization), I get (system:9) Bad file descriptor when performing write/read operations. While trying to figure out the problem, I went back for the original computer and by looking at /proc/XXXX/maps I noticed that my executable was actually loading the default boost libraries under /usr/lib at runtime, despite being linked with the custom-built versions at compilation.
Thus my issues are 1) how to make the system prioritize /usr/local/lib over /usr/lib or force it to identify the _boost_oldabi suffix as different libraries; and 2) figure out why does adding the GCC ABI macro seems to break the code; The source code, make files and libraries are exactly the same, the only thing that differs between the computers is the OS (Arch Linux and Ubuntu).
Preamble: If you need your custom boost libraries only for a single tool, it is better to install them into a place which is specific to this single tool. Installing custom libs into a system-wide location (/usr/local/lib) is not a very good idea, because
It increases the risk that also your system-wide apps will use the bad one.
If you want once cleanup /usr/local/lib, you won't be able to know, what is using the libs there.
It would be better if you would make them available only for your software. For example, if your software is in /srv/niceproject/bin, then the libs could exist in /srv/niceproject/lib.
Note, the shared library handling is enough smart in Linux to handle libraries with different versions available, all the binaries will map in the libs according to their needed versions. But, it depends on special symbols inserted by the linker into the binaries. If you change only the compiler flags, these symbols may be the same, thus a false lib may be linked in.
You can prioritize /usr/local/lib by changing the shared library order in /etc/ld.so.conf and in /etc/ld.so.conf.d, and then calling an ldconfig. To get the list of the currently visible libraries, use ldconfig -p.
There is a lot, so an ldconfig -p|grep libboost could be a nicer result.
The shared library directory order is affected also by the LD_LIBRARY_PATH environment variable, what can be specified for every process. Thus, calling your tool with
LD_LIBRARY_PATH=/srv/niceproject/lib /srv/niceproject/bin/mytool
will call mytool on a way, that it tries to find the libs in /srv/niceproject/lib first, and looks the system-wide ones only after it. It is a colon-separated list of pathes, like $PATH.
Linking your binary with a library given by an absolute path will hardcode the absolute path into the library. So:
gcc -o mytool -l/usr/local/lib/libboost_oldapi_anything.so
Also this is a dirty solution in general, but it would solve your problem.
You can see, what exactly an actual library is loading, with the ldd tool:
# ldd /bin/bash
linux-vdso.so.1 => (0x00007ffd1c1cf000)
libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f4baa499000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4baa295000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4ba9eca000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4baa6d0000)
ldd is a special binary, you can specify any LD_LIBRARY_PATH, it won't affect it (but it will affect the binary you are checking with it). In my example, with an LD_LIBRARY_PATH=/srv/niceproject/lib ldd /srv/niceproject/bin/mytool will show, how the mytool is affected by this environment variable.

Error while loading shared libraries:libmc_data.so.1

I can not run C++ code,because I got
Error while loading shared libraries:libmc_data.so.1:cannot open shared object file:No such file or directory
But with ls I got
milenko#milenko-HP-Compaq-6830s:~/ProcMT64/old/version_september_2015/bin$ ls -l libmc_data.so.1
lrwxrwxrwx 1 milenko milenko 19 Set 19 07:36 libmc_data.so.1 -> libmc_data.so.1.0.0
What's the problem here?
Normally the linux loader will look for libraries in several predefined locations plus the directories listed in the env variable LD_LIBRARY_PATH (man ld for more information). Try adding the directory ~/ProcMT64/old/version_september_2015/bin to your LD_LIBRARY_PATH by running:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/ProcMT64/old/version_september_2015/bin
and run your application, it should work (unless other libraries are missing ..)

Building boost-python example

I'm running Ubuntu 13.10 . I installed libboost1.54-dev.
I did a Git checkout of Boost, and did a checkout to "boost-1.54.0".
I changed directories to boost/libs/python/example/tutorial in the source.
I ran "bjam". I get:
$ bjam
Unable to load Boost.Build: could not find build system.
---------------------------------------------------------
/home/dustin/build/boost/libs/python/example/boost-build.jam attempted to load the build system by invoking
'boost-build ../../../tools/build/v2 ;'
but we were unable to find "bootstrap.jam" in the specified directory
or in BOOST_BUILD_PATH (searching /home/dustin/build/boost/libs/python/example/../../../tools/build/v2, /usr/share/boost-build).
Please consult the documentation at 'http://www.boost.org'.
There are only three files in the example directory:
-rw-r--r-- 1 dustin dustin 484 Mar 1 12:59 hello.cpp
-rwxr-xr-x 1 dustin dustin 275 Mar 1 12:59 hello.py
-rw-r--r-- 1 dustin dustin 1445 Mar 1 15:43 Jamroot
The directions say that it should be just that easy: http://www.boost.org/doc/libs/1_54_0/libs/python/doc/tutorial/doc/html/python/hello.html
The last few lines of the strace is:
openat(AT_FDCWD, "/home/dustin/build/boost/libs/python/example/tutorial", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/dustin/build/boost/libs/python/example", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/share/boost-build", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
open("/home/dustin/build/boost/libs/python/example/boost-build.jam", O_RDONLY) = 3
Why is it looking for boost-build.jam? What am I missing?
You can create your own boost-build.jam. For quickstart example (which is broke) just create a file called boost-build.jam with this in it and make sure it points to the src directory. It is discussed here http://lists.boost.org/boost-build/2014/11/27738.php
# Edit this path to point at the tools/build/v2 subdirectory of your
# Boost installation. Absolute paths work, too.
boost-build ../../../../tools/build/src ;
In essence, bjam is an interpreter, and Boost.Build is a build system written in bjam files. When bjam starts, it will attempt to locate the jam files for Boost.Build. In this case, bjam attempted to locate boost-build.jam relative to the tutorial and errors when it is missing. To build the tutorial, either:
Verify that the boost/tools/build submodule has been initialized from within the boost git repository. Boost.Python has other dependencies, so it may be easier to initialize all submodules. This will allow the bjam interpreter installed from the libboost1.54-dev package to locate Boost.Build from within the repository, and build the tutorial and its dependencies.
To build against packaged libraries:
Install the libboost1.54 package. This will install the Boost.Python shared library and its dependencies.
Modify the tutorial's Jamroot file. It should no longer attempt to use the boost project, and should explicitly list the Boost.Python shared library path:
-# Specify the path to the Boost project. If you move this project,
-# adjust this path to refer to the Boost root directory.
-use-project boost
- : ../../../.. ;
-
# Set up the project-wide requirements that everything uses the
-# boost_python library from the project whose global ID is
-# /boost/python.
+# boost_python library.
project
- : requirements <library>/boost/python//boost_python ;
+ : requirements <library>/usr/lib/libboost_python-py27.so ;
The library path and name may need to be changed based on where the libboost-python1.54-dev packaged installed the Boost.Python library.
Set the BOOST_BUILD_PATH environment variable to /usr/share/boost-build/kernel or wherever the libboost1.54-dev package installed boost-build.jam.

Cyassl-2.4.2 missing libcyassl.a file

I'm have built and installed Cyassl-2.4.2 on my unix machine to use with a project of mine; however, when I try to compile my c project, the compiler cannot find the libcyassl.a file. I have searched many times in the Cyassl.2.4.2 folder and am unable to find the file myself. Where is this file locate? Have I missed a step in bulding/installing Cyassl.2.4.2? Thanks!
I had successfully built and installed CyaSSL 2.4.2 here and this is what I got:
~/cyassl$ find . -iname "*.a"
./src/.libs/libcyassl.a
~/cyassl$ ls /usr/local/lib/
libcyassl.a libcyassl.la libcyassl.so libcyassl.so.3 libcyassl.so.3.0.3
The building process used was:
./autogen.sh
./configure
make
sudo make install
The installation prints out:
Libraries have been installed in:
/usr/local/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the -LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to theLD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the LD_RUN_PATH' environment variable
during linking
- use the-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.