shared object library not found when running program, but it's linked during compiling - c++

Update: issue is solved. The library was Made for Armv7a CPUs but it was "soft float" Not "hard float". It seems like my machine is HF and Not SF compatible
My program depends on an externally build .so library called libMyLib.so. When I compile the program like this:
$ g++ -std=c++11 main.cpp -o run -pthread
it reports that there are a lot of undefined references, obviously because i didn't include libMyLib.so when compiling. So the compiler knows what he needs to compile the program. When i compile the program like this:
$ g++ -std=c++11 main.cpp -o run -pthread -lMyLib
it doesn't report any errors and creates the file "run". Notice that libMyLib.so is already in /usr/local/lib and it looks like it is linked when compiling since the references are defined now and the "run" file is created. But as i run the file, this happens:
$ ./run
./run: error while loading shared libraries: libMyLib.so: cannot open shared object file: No such file or directory
I've checked with ldd and it shows me this:
$ ldd run
...
libMyLib.so => not found
...
So ldd doesn't find the library on execution, but it finds it while compiling. I'm quite new to Linux and linking libraries so i don't know what to do.
Also, running ldd on the .so file returns this:
$ ldd /usr/local/lib/libMyLib.so
not a dynamic executable
I've already checked an this message may occur when running a .so file on the wrong platform. But i've checked, the library is compiled for arm (I'm running on a raspberry pi -> arm):
$ objdump -f /usr/local/lib/libMyLib.so | grep ^architecture
architecture: arm, flags 0x00000150:
I also update the linker:
$ sudo ldconfig -v
...
/usr/local/lib:
libwiringPi.so -> libwiringPi.so.2.44
libwiringPiDev.so -> libwiringPiDev.so.2.44
libMyLib.so -> libMyLib.so.1
...
I really have no clue why this might still happen. Can anyone help me?

/usr/local/lib is one of the directories that the linker searches by default
for libraries specified with the -l option, so your linkage succeeds.
At runtime however, the program loader by default searches for the linked
libraries in:-
/lib, /usr/lib and among the libraries whose names and locations have been cached in the ldconfig cache, /etc/ld.so.cache.
The directories listed in the value of the environment variable LD_LIBRARY_PATH,
in the current shell.
The ldconfig cache is only updated when ldconfig is run. See man ldconfig.
The loader fails to find libMyLib.so at runtime because you have not
run ldconfig since you placed that library in /usr/local/lib and
neither have you correctly added /usr/local/lib to the LD_LIBRARY_PATH
in the same shell in which you try to run the program.
It is inconvenient and otherwise undesirable to require a special setting of
LD_LIBRARY_PATH to enable a program to run.
To enable the loader to find your library, run ldconfig as root. This
will succeed provided that /usr/local/lib is listed in /etc/ld.so.conf,
or in one of the files included by /etc/ld.so.conf. If it's not, then
you can explicitly cache the shared libraries from /usr/local/lib by running
ldconfig /usr/local/lib, as root.

First check LD_LIBRARY_PATH variable is having the path to your library directory
$ echo $LD_LIBRARY_PATH
If not there then update the library path.
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/library
Use strace to debug.
strace -f ./run

Related

Does -rpath and $LD_LIBRAY_PATH has same functionality?

The book Advanced Linux Programming contains:
The system searches only /lib and /usr/lib, by default. If a shared library that is linked into your program is installed outside those directories, it will not be found, and the system will refuse to run the program.
One solution to this problem is to use the -Wl, -rpath option when linking the program. Another solution to this problem is to set the LD_LIBRARY_PATH environment variable when running the program.
So from information above, I assume adding path to $LD_LIBRARY_PATH is equivalent to passing argument to linker when compiling something like -Wl,-rpath,SOME_PATH until I hit this problem:
I am trying to generate a shared library and this shared library depends on some .so files somewhere at my system. Here's the command for compilation:
g++ -fPIC -shared -Wl,-soname,libA.so -o libA.so SRC.cpp -L PATH_A -I PATH_B -Wl,-rpath,PATH_C:PATH_D:PATH_E -la -lb -lc
After I compile, I use ldd to check if there's any dependency issue and something unexpected to me happens:
Some of dependency are found and some of them are not (libA.so => not found), however all the paths it needs (PATH_C PATH_DPATH_E) have been passed to argument -Wl,-rpath.
So it seems -Wl,-rpath does not work for this case , the only solution to make it work is to add the path to libA.so to $LD_LIBRARY_PATH and ldd returns that all the dependency are found.
I would like to know why -Wl,-rpath fails in, what's the difference between adding the path to $LD_LIBRARY_PATH and passing an argument to -Wl,-rpath?

Link libraries to linux biinary file in c++

I'm compiling a c++ program using g++ and i am using two libraries called libsdl2-dev and libsdl2-image-dev
I installed both these libraries in my ubuntu machine with the commands
apt install libsdl2-dev libsdl2-image-dev and when I compile the program everything works fine. Then I copied these libraries from /usr/lib/x86_64-linux-gnu/ to my working dir with the binary file to be able to give this folder to someone else.
The problem comes when the user that hasn't installed these libraries tries to open my program by writing ./main (the binary file). Since he hasn't installed these libraries he would get an error like "can't open shared object: no such file or directory".
This happens because the binary file looks for these libraries in /usr/lib etc...
What i need
I need that my binary file looks for these libraries in the same folder,and not in /usr/lib/x86 etc.., from what I read I have to do something like rpath
The IDE used is Sublime Text and the syntax used to compile all my files is this:
g++ -c src/*.cpp -std=c++14 -m64 -g -Wall -I include && g++ *.o -o bin/debug/main -lSDL2main -lSDL2 -lSDL2_image && ./bin/debug/main`
Structure of folders
I got the project dir with and inside that i got 4 more directories, each one called: bin (with the debug subdirectory, where we got the final compile), include (with hpp files), res (with all textures), and src with all cpp files to compile, the other files are project files and .o files
I'm using Ubuntu 20.04-2 LTS and the same is for the other user's PC
Thanks in advance for any help!
That's because the dynamic linker loading runtime dependencies looks for them in some specified locations, which are "by default" your system library directories (where those libraries got installed by apt).
The other user should ideally install those libraries too (which could be done "automatically" if you build a .deb package with proper dependencies)
Otherwise you would have to change the runpath of your program by adding -Wl,-rpath='$ORIGIN', which makes the dynamic linker look for dependencies just where the binary is located.
$ORIGIN here is a special variable meaning "this executable" which is what you wanted to achieve.
see rpath
and A description of RPATH $ORIGIN
I found a way to resolve!
I used the program patchelf to add an rpath to my directory (linked to the binary file) now everything works
use ldd ./bin/debug/main to check the library
export LD_LIBRARY_PATH =$LD_LIBRARY_PATH:"your library path"
run the program,if this is not work. use patchelf to change the rpath to you r library

A simpler method to load shared libraries without root

I am trying to compile and run a C++ program on a server where I don't have root access. I am having trouble linkingboost_iostreams library.
I can successfully compile my program by pointing to the boost installation directory using the -L flag as:
g++ -I path/to/boost/build/include -o out prog1.cpp prog2.cpp -L path/to/boost/build/lib -lboost_iostreams
However, if I run the program as ./out I get the error error while loading shared libraries: libboost_iostreams.so.1.67.0: cannot open shared object file: No such file or directory since the linker is not able to locate libboost_iostreams.so.1.67.0 (which DOES exist under path/to/boost/build/lib)
Thanks to this answer, I was able to explicitly specify LD_LIBRARY_PATH and run the program as
LD_LIBRARY_PATH="path/to/boost/build/lib" ./out.
Since I am not root, I can't run ldconfig either. I was wondering if there is a way to load dynamic libraries without having to prefix LD_LIBRARY_PATH when you run the program and no root access.
I have figured out a way to solve this using the method explained here https://amir.rachum.com/blog/2016/09/17/shared-libraries/. The solution is to use rpath during compilation.
According to the article The only difference between rpath and
runpath is the order they are searched in. Specifically, their
relation to LD_LIBRARY_PATH - rpath is searched in before
LD_LIBRARY_PATH while runpath is searched in after. The meaning of
this is that rpath cannot be changed dynamically with environment
variables while runpath can.
In short once you compile with -rpath path/to/boost/build/lib, the directory containing the library libboost_iostreams.so.1.67.0 is searched at runtime without having to prefix LD_LIBRARY_PATH="path/to/boost/build/lib" ./out.
After compiling with
g++ -I path/to/boost/build/include -o out prog1.cpp prog2.cpp -L path/to/boost/build/lib -lboost_iostreams -rpath path/to/boost/build/lib
I was able to run ./out without any issues.
EDIT 1
As pointed by Nikos in the comments, alternatively you can set your LD_LIBRARY_PATH by export LD_LIBRARY_PATH=path/to/boost/build/lib. Add this line to .~/.bashrc file so that it is not lost when you log out.

g++ compiling: can I link to a directory with symlinked binaries?

The below compiles:
g++ -L../../lib -o my_prog my_prog.cpp -ltest1 -ltest2
where ../../lib contains symlinks to libtest1.so and libtest2.so
But I am getting an error when I run the program: "error while loading shared libraries: libtest1.so: cannot open shared object file: No such file or directory" and am not sure if symlinking is the culprit.
Option -L is for linker ld to find .a and .so during linking.
Option -Wl,-rpath= is for the dynamic linker ld.so to find .so when the application is run. You need to use -Wl,-rpath= when a required shared library is not in (standard system) directories specified in /etc/ld.so.conf.
Use $ORIGIN dynamic linker variable to make rpath relative to the path of your executable:
g++ -L../../lib -Wl,-rpath='${ORIGIN}/../../lib' -o my_prog my_prog.cpp -ltest1 -ltest2
Be careful to make sure ${ORIGIN} is not expanded by the shell or your makefile (this is why it is in single quotes).
${ORIGIN}:
$ORIGIN and rpath
ld.so understands the string $ORIGIN (or equivalently ${ORIGIN}) in an rpath specification (DT_RPATH or DT_RUNPATH) to mean the directory containing the application executable. Thus, an application located in somedir/app could be compiled with gcc -Wl,-rpath,'$ORIGIN/../lib' so that it finds an associated shared library in somedir/lib no matter where somedir is located in the directory hierarchy. This facilitates the creation of "turn-key" applications that do not need to be installed into special directories, but can instead be unpacked into any directory and still find their own shared libraries.
What happens at runtime is related to the rpath.
You may want (not really recommended, see this) to set your LD_LIBRARY_PATH appropriately before running your executable, or better yet you want to set the executable's rpath when linking it (e.g. by passing -Wl,--rpath $(realpath ../../lib/) to the g++ command doing the link.
Read Drepper's How to write shared libraries paper and the Program Library HowTo

Executing cross-compiled C++ program using Boost on Raspberry Pi

I have built a GCC cross toolchain for the RPi and can cross-compile C++ source and successfully run it after copying the executable to the RPi.
Next I built the Boost libraries targeting ARM, using the cross toolchain. I can successfully build and link C++ source to those Boost libraries using the cross toolchain on my PC.
I then copied the program, dynamically linked to Boost, to the RPi and copied all built libraries into /usr/local/lib on the Pi. However, executing fails:
$ ./my_program
./my_program: error while loading shared libraries: libboost_system.so.1.60.0: cannot open shared object file: No such file or directory
Again, this library, libboost_system.so.1.60.0, exists in /usr/local/lib.
I also tried
export LD_LIBRARY_PATH='/usr/local/lib'
but that doesn't change anything. What am I doing wrong?
EDIT:
I build all source files like this (rpi-g++ is a symlink to my cross-compiler):
rpi-g++ -c -std=c++1y -Wall -Wextra -pedantic -O2 -I /path/to/cross/boost/include *.cpp
rpi-g++ -o myprog *.o -L /path/to/cross/boost/lib/ -lboost_system -pthread
EDIT 2:
When linked with
rpi-g++ -o myprog *.o -L /path/to/cross/boost/lib/ -rdynamic -lboost_system -pthread
the problem remains the same. I have checked and verified everything suggested by Technaton as well. Strangely, ldd insists that the created executable is "not a dynamic executable" (checked that on my PC and on the RPi), which doesn't make sense to me.
There are several things you can check. I've posted a complete check list here, but judging from your linker command line, number 5 is probably the culprit.
Check that your library and your program are correctly build for the target architecture. You can verify that by using file ./myprog and file libboost_system.so.1.60.0.
Make sure that you have copied the actual shared object, and not a link to it.
Ensure that the shared object file's permissions are sane (0755).
Run ldconfig -v and check that your shared object file is picked up. Normally, /usr/local/lib is in the standard library search path, and LD_LIBRARY_PATH is not required.
Make sure that your program is actually dynamically linked by running ldd ./myprog. Judging from your linker command line, that is the problem: You're missing -rdynamic.
Check the paths returned from ldd: If you have linked with rpath, the library search path might be screwed up. Try again without -rpath.