I am trying to compile and run a C++ program on a server where I don't have root access. I am having trouble linkingboost_iostreams library.
I can successfully compile my program by pointing to the boost installation directory using the -L flag as:
g++ -I path/to/boost/build/include -o out prog1.cpp prog2.cpp -L path/to/boost/build/lib -lboost_iostreams
However, if I run the program as ./out I get the error error while loading shared libraries: libboost_iostreams.so.1.67.0: cannot open shared object file: No such file or directory since the linker is not able to locate libboost_iostreams.so.1.67.0 (which DOES exist under path/to/boost/build/lib)
Thanks to this answer, I was able to explicitly specify LD_LIBRARY_PATH and run the program as
LD_LIBRARY_PATH="path/to/boost/build/lib" ./out.
Since I am not root, I can't run ldconfig either. I was wondering if there is a way to load dynamic libraries without having to prefix LD_LIBRARY_PATH when you run the program and no root access.
I have figured out a way to solve this using the method explained here https://amir.rachum.com/blog/2016/09/17/shared-libraries/. The solution is to use rpath during compilation.
According to the article The only difference between rpath and
runpath is the order they are searched in. Specifically, their
relation to LD_LIBRARY_PATH - rpath is searched in before
LD_LIBRARY_PATH while runpath is searched in after. The meaning of
this is that rpath cannot be changed dynamically with environment
variables while runpath can.
In short once you compile with -rpath path/to/boost/build/lib, the directory containing the library libboost_iostreams.so.1.67.0 is searched at runtime without having to prefix LD_LIBRARY_PATH="path/to/boost/build/lib" ./out.
After compiling with
g++ -I path/to/boost/build/include -o out prog1.cpp prog2.cpp -L path/to/boost/build/lib -lboost_iostreams -rpath path/to/boost/build/lib
I was able to run ./out without any issues.
EDIT 1
As pointed by Nikos in the comments, alternatively you can set your LD_LIBRARY_PATH by export LD_LIBRARY_PATH=path/to/boost/build/lib. Add this line to .~/.bashrc file so that it is not lost when you log out.
Related
The book Advanced Linux Programming contains:
The system searches only /lib and /usr/lib, by default. If a shared library that is linked into your program is installed outside those directories, it will not be found, and the system will refuse to run the program.
One solution to this problem is to use the -Wl, -rpath option when linking the program. Another solution to this problem is to set the LD_LIBRARY_PATH environment variable when running the program.
So from information above, I assume adding path to $LD_LIBRARY_PATH is equivalent to passing argument to linker when compiling something like -Wl,-rpath,SOME_PATH until I hit this problem:
I am trying to generate a shared library and this shared library depends on some .so files somewhere at my system. Here's the command for compilation:
g++ -fPIC -shared -Wl,-soname,libA.so -o libA.so SRC.cpp -L PATH_A -I PATH_B -Wl,-rpath,PATH_C:PATH_D:PATH_E -la -lb -lc
After I compile, I use ldd to check if there's any dependency issue and something unexpected to me happens:
Some of dependency are found and some of them are not (libA.so => not found), however all the paths it needs (PATH_C PATH_DPATH_E) have been passed to argument -Wl,-rpath.
So it seems -Wl,-rpath does not work for this case , the only solution to make it work is to add the path to libA.so to $LD_LIBRARY_PATH and ldd returns that all the dependency are found.
I would like to know why -Wl,-rpath fails in, what's the difference between adding the path to $LD_LIBRARY_PATH and passing an argument to -Wl,-rpath?
Update: issue is solved. The library was Made for Armv7a CPUs but it was "soft float" Not "hard float". It seems like my machine is HF and Not SF compatible
My program depends on an externally build .so library called libMyLib.so. When I compile the program like this:
$ g++ -std=c++11 main.cpp -o run -pthread
it reports that there are a lot of undefined references, obviously because i didn't include libMyLib.so when compiling. So the compiler knows what he needs to compile the program. When i compile the program like this:
$ g++ -std=c++11 main.cpp -o run -pthread -lMyLib
it doesn't report any errors and creates the file "run". Notice that libMyLib.so is already in /usr/local/lib and it looks like it is linked when compiling since the references are defined now and the "run" file is created. But as i run the file, this happens:
$ ./run
./run: error while loading shared libraries: libMyLib.so: cannot open shared object file: No such file or directory
I've checked with ldd and it shows me this:
$ ldd run
...
libMyLib.so => not found
...
So ldd doesn't find the library on execution, but it finds it while compiling. I'm quite new to Linux and linking libraries so i don't know what to do.
Also, running ldd on the .so file returns this:
$ ldd /usr/local/lib/libMyLib.so
not a dynamic executable
I've already checked an this message may occur when running a .so file on the wrong platform. But i've checked, the library is compiled for arm (I'm running on a raspberry pi -> arm):
$ objdump -f /usr/local/lib/libMyLib.so | grep ^architecture
architecture: arm, flags 0x00000150:
I also update the linker:
$ sudo ldconfig -v
...
/usr/local/lib:
libwiringPi.so -> libwiringPi.so.2.44
libwiringPiDev.so -> libwiringPiDev.so.2.44
libMyLib.so -> libMyLib.so.1
...
I really have no clue why this might still happen. Can anyone help me?
/usr/local/lib is one of the directories that the linker searches by default
for libraries specified with the -l option, so your linkage succeeds.
At runtime however, the program loader by default searches for the linked
libraries in:-
/lib, /usr/lib and among the libraries whose names and locations have been cached in the ldconfig cache, /etc/ld.so.cache.
The directories listed in the value of the environment variable LD_LIBRARY_PATH,
in the current shell.
The ldconfig cache is only updated when ldconfig is run. See man ldconfig.
The loader fails to find libMyLib.so at runtime because you have not
run ldconfig since you placed that library in /usr/local/lib and
neither have you correctly added /usr/local/lib to the LD_LIBRARY_PATH
in the same shell in which you try to run the program.
It is inconvenient and otherwise undesirable to require a special setting of
LD_LIBRARY_PATH to enable a program to run.
To enable the loader to find your library, run ldconfig as root. This
will succeed provided that /usr/local/lib is listed in /etc/ld.so.conf,
or in one of the files included by /etc/ld.so.conf. If it's not, then
you can explicitly cache the shared libraries from /usr/local/lib by running
ldconfig /usr/local/lib, as root.
First check LD_LIBRARY_PATH variable is having the path to your library directory
$ echo $LD_LIBRARY_PATH
If not there then update the library path.
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/library
Use strace to debug.
strace -f ./run
I installed boost_1_55_0 under /usr/local/boost_1_55_0, and in my makefile I set the linking flag as:
CC = g++
CFLAGS = -I ./ -I/usr/local/boost_1_55_0 -w -std=c++11
LFLAGS = -L./ -lm -lpthread -L/usr/local/boost_1_55_0/lib -lboost_regex
I encountered nothing during compile time, but I got:
./$(exec): error while loading shared libraries:
libboost_regex.so.1.55.0: cannot open shared object file: No such file
or directory
I'm pretty sure I cleaned all old files and compiled again, I also granted access to read the files in usr/local/boost_1_55_0 by sudo chmod -R +x /usr/local/boost_1_55_0 just in case I banned access to read. However, I still got this error. I also tried to move boost_1_55_0 in /usr/lib/ but it does not work either.
I've also read this question: Compiling issue with boost and tried both solutions under this question, but neither of them works for me. Can someone figure out a different solution?
As the error implies, your executable is unable to find the shared library. Add the .so file (or all of them, if you want to be safe) that was generated by the boost build process to the same directory where your program's executable is getting generated, and try running the program again.
I manually cleaned the executable files and compiled it again, and it somehow solved my problem. I am guessing there must be some config in my system messed up so that the executable would invisibly linked to some outdated directory and cause this problem.
I have built a GCC cross toolchain for the RPi and can cross-compile C++ source and successfully run it after copying the executable to the RPi.
Next I built the Boost libraries targeting ARM, using the cross toolchain. I can successfully build and link C++ source to those Boost libraries using the cross toolchain on my PC.
I then copied the program, dynamically linked to Boost, to the RPi and copied all built libraries into /usr/local/lib on the Pi. However, executing fails:
$ ./my_program
./my_program: error while loading shared libraries: libboost_system.so.1.60.0: cannot open shared object file: No such file or directory
Again, this library, libboost_system.so.1.60.0, exists in /usr/local/lib.
I also tried
export LD_LIBRARY_PATH='/usr/local/lib'
but that doesn't change anything. What am I doing wrong?
EDIT:
I build all source files like this (rpi-g++ is a symlink to my cross-compiler):
rpi-g++ -c -std=c++1y -Wall -Wextra -pedantic -O2 -I /path/to/cross/boost/include *.cpp
rpi-g++ -o myprog *.o -L /path/to/cross/boost/lib/ -lboost_system -pthread
EDIT 2:
When linked with
rpi-g++ -o myprog *.o -L /path/to/cross/boost/lib/ -rdynamic -lboost_system -pthread
the problem remains the same. I have checked and verified everything suggested by Technaton as well. Strangely, ldd insists that the created executable is "not a dynamic executable" (checked that on my PC and on the RPi), which doesn't make sense to me.
There are several things you can check. I've posted a complete check list here, but judging from your linker command line, number 5 is probably the culprit.
Check that your library and your program are correctly build for the target architecture. You can verify that by using file ./myprog and file libboost_system.so.1.60.0.
Make sure that you have copied the actual shared object, and not a link to it.
Ensure that the shared object file's permissions are sane (0755).
Run ldconfig -v and check that your shared object file is picked up. Normally, /usr/local/lib is in the standard library search path, and LD_LIBRARY_PATH is not required.
Make sure that your program is actually dynamically linked by running ldd ./myprog. Judging from your linker command line, that is the problem: You're missing -rdynamic.
Check the paths returned from ldd: If you have linked with rpath, the library search path might be screwed up. Try again without -rpath.
I am trying to build and link a C++, cmake-based project with clang (3.0). This project links to several libraries that are installed in a custom directory /my/dir/. This directory is included in the LD_LIBRARY_PATH and LIBRARY_PATH environment variables. Project builds and links fine with g++.
The link command generated and executed by cmake looks like the following:
/usr/bin/clang++ -O3 stuff.cpp.o -o stuff -rdynamic -lmylib
ld then complains with the following message:
/usr/bin/ld: cannot find -lmylib
The link command above runs fine whenever I manually add -L/my/dir/. Is there a way to link without specifying the -L flag?
The $LD_LIBRARY_PATH environment variable (and its various alternatives on other UNIX-based platforms) is used at runtime, not link time, to find libraries.
Using -L is the correct approach and cannot be avoided.
Note: A better approach under Linux (you don't specify your platform so I'm guessing) is to correctly configure a file in /etc/ld.so.conf.d/ and avoid using $LD_LIBRARY_PATH altogether.