We created a custom shared library from some C++ code using
g++ -c -fPIC customTest.cpp
g++ -shared -o libcustomTest.so customTest.o
And we put it in the project directory and in our makefile we have the default target being
main: main.o
nvcc $^ -o main -lcustomTest -L.
And this works just fine.
The problem is, we'd like to move our library to /usr/lib/ or any arbitrary folder and still have the program locate it and use it but this hasn't been happening as we want it to.
We have a folder in our root that we created called libTest and in that folder we put our library customTest.so.0.1. Then we edited ld.so.conf in /etc/ to had /libTest in it.
Then we went to the directory of our program files and ran
ldconfig -v
which looked like this but larger
libpanel.so.5 -> libpanel.so.5.9
libt1.so.5 -> libt1.so.5.1.2
libbluetooth.so.3 -> libbluetooth.so.3.11.4
libgck-1.so.0 -> libgck-1.so.0.0.0
libdca.so.0 -> libdca.so.0.0.0
a lot of links are created and what not, but libcustomTest.so is not one of them. Not surprisingly, when we run make the custom library can't be located.
Can anyone point us in the right direction with what we are doing wrong? By the way we are on Ubuntu 11.10
g++ -shared -o libcustomTest.so customTest.o
This will create a shared library with SONAME not set.
We have a folder in our root that we created called libTest and in that folder we put our library customTest.so.0.1
Don't do that. Just copy libcustomTest.so into /libTest, and be done with it.
a lot of links are created and what not, but libcustomTest.so is not one of them.
That's expected result. ldconfig creates symlinks from SONAME to actual implementation binary. Since you didn't set SONAME, no symlink for you.
Unless you understand what SONAME is for, don't bother setting it (via -Wl,--soname=... flag) either. On Linux, SONAME and external library versioning is usually the wrong answer, as symbol versioning provides much better approach anyway.
Related
I am trying to compile and run a C++ program on a server where I don't have root access. I am having trouble linkingboost_iostreams library.
I can successfully compile my program by pointing to the boost installation directory using the -L flag as:
g++ -I path/to/boost/build/include -o out prog1.cpp prog2.cpp -L path/to/boost/build/lib -lboost_iostreams
However, if I run the program as ./out I get the error error while loading shared libraries: libboost_iostreams.so.1.67.0: cannot open shared object file: No such file or directory since the linker is not able to locate libboost_iostreams.so.1.67.0 (which DOES exist under path/to/boost/build/lib)
Thanks to this answer, I was able to explicitly specify LD_LIBRARY_PATH and run the program as
LD_LIBRARY_PATH="path/to/boost/build/lib" ./out.
Since I am not root, I can't run ldconfig either. I was wondering if there is a way to load dynamic libraries without having to prefix LD_LIBRARY_PATH when you run the program and no root access.
I have figured out a way to solve this using the method explained here https://amir.rachum.com/blog/2016/09/17/shared-libraries/. The solution is to use rpath during compilation.
According to the article The only difference between rpath and
runpath is the order they are searched in. Specifically, their
relation to LD_LIBRARY_PATH - rpath is searched in before
LD_LIBRARY_PATH while runpath is searched in after. The meaning of
this is that rpath cannot be changed dynamically with environment
variables while runpath can.
In short once you compile with -rpath path/to/boost/build/lib, the directory containing the library libboost_iostreams.so.1.67.0 is searched at runtime without having to prefix LD_LIBRARY_PATH="path/to/boost/build/lib" ./out.
After compiling with
g++ -I path/to/boost/build/include -o out prog1.cpp prog2.cpp -L path/to/boost/build/lib -lboost_iostreams -rpath path/to/boost/build/lib
I was able to run ./out without any issues.
EDIT 1
As pointed by Nikos in the comments, alternatively you can set your LD_LIBRARY_PATH by export LD_LIBRARY_PATH=path/to/boost/build/lib. Add this line to .~/.bashrc file so that it is not lost when you log out.
Update: issue is solved. The library was Made for Armv7a CPUs but it was "soft float" Not "hard float". It seems like my machine is HF and Not SF compatible
My program depends on an externally build .so library called libMyLib.so. When I compile the program like this:
$ g++ -std=c++11 main.cpp -o run -pthread
it reports that there are a lot of undefined references, obviously because i didn't include libMyLib.so when compiling. So the compiler knows what he needs to compile the program. When i compile the program like this:
$ g++ -std=c++11 main.cpp -o run -pthread -lMyLib
it doesn't report any errors and creates the file "run". Notice that libMyLib.so is already in /usr/local/lib and it looks like it is linked when compiling since the references are defined now and the "run" file is created. But as i run the file, this happens:
$ ./run
./run: error while loading shared libraries: libMyLib.so: cannot open shared object file: No such file or directory
I've checked with ldd and it shows me this:
$ ldd run
...
libMyLib.so => not found
...
So ldd doesn't find the library on execution, but it finds it while compiling. I'm quite new to Linux and linking libraries so i don't know what to do.
Also, running ldd on the .so file returns this:
$ ldd /usr/local/lib/libMyLib.so
not a dynamic executable
I've already checked an this message may occur when running a .so file on the wrong platform. But i've checked, the library is compiled for arm (I'm running on a raspberry pi -> arm):
$ objdump -f /usr/local/lib/libMyLib.so | grep ^architecture
architecture: arm, flags 0x00000150:
I also update the linker:
$ sudo ldconfig -v
...
/usr/local/lib:
libwiringPi.so -> libwiringPi.so.2.44
libwiringPiDev.so -> libwiringPiDev.so.2.44
libMyLib.so -> libMyLib.so.1
...
I really have no clue why this might still happen. Can anyone help me?
/usr/local/lib is one of the directories that the linker searches by default
for libraries specified with the -l option, so your linkage succeeds.
At runtime however, the program loader by default searches for the linked
libraries in:-
/lib, /usr/lib and among the libraries whose names and locations have been cached in the ldconfig cache, /etc/ld.so.cache.
The directories listed in the value of the environment variable LD_LIBRARY_PATH,
in the current shell.
The ldconfig cache is only updated when ldconfig is run. See man ldconfig.
The loader fails to find libMyLib.so at runtime because you have not
run ldconfig since you placed that library in /usr/local/lib and
neither have you correctly added /usr/local/lib to the LD_LIBRARY_PATH
in the same shell in which you try to run the program.
It is inconvenient and otherwise undesirable to require a special setting of
LD_LIBRARY_PATH to enable a program to run.
To enable the loader to find your library, run ldconfig as root. This
will succeed provided that /usr/local/lib is listed in /etc/ld.so.conf,
or in one of the files included by /etc/ld.so.conf. If it's not, then
you can explicitly cache the shared libraries from /usr/local/lib by running
ldconfig /usr/local/lib, as root.
First check LD_LIBRARY_PATH variable is having the path to your library directory
$ echo $LD_LIBRARY_PATH
If not there then update the library path.
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/library
Use strace to debug.
strace -f ./run
When using the -l flag in a c++ makefile, I couldn't find information on what steps gcc takes in searching for that library.
For clarity of what I'm asking, I'll make up an example answer (this is completely made up):
First searches the -L flag directory
Next searches the /lib directory
Something something etc
Some extra info:
The reason I'm stuck in this is because I currently have a program that uses libconfig in the makefile like so:
g++ $(CFLAGS) -o libvldsk_config.so $(SRCDIR)/config.cpp -Wall -shared -fPIC -I$(SRCDIR) -I$(COMMON_SRCDIR) -lconfig
but this was made assuming that libconfig is installed via a package manager. I installed via source because the linux version was too old to have it in a package manager. And so I'm trying to modify my program that uses libconfig to search for the libconfig .so file in the system's /lib directory, and if it doesn't exist search for it in the current project directory.
I have built a GCC cross toolchain for the RPi and can cross-compile C++ source and successfully run it after copying the executable to the RPi.
Next I built the Boost libraries targeting ARM, using the cross toolchain. I can successfully build and link C++ source to those Boost libraries using the cross toolchain on my PC.
I then copied the program, dynamically linked to Boost, to the RPi and copied all built libraries into /usr/local/lib on the Pi. However, executing fails:
$ ./my_program
./my_program: error while loading shared libraries: libboost_system.so.1.60.0: cannot open shared object file: No such file or directory
Again, this library, libboost_system.so.1.60.0, exists in /usr/local/lib.
I also tried
export LD_LIBRARY_PATH='/usr/local/lib'
but that doesn't change anything. What am I doing wrong?
EDIT:
I build all source files like this (rpi-g++ is a symlink to my cross-compiler):
rpi-g++ -c -std=c++1y -Wall -Wextra -pedantic -O2 -I /path/to/cross/boost/include *.cpp
rpi-g++ -o myprog *.o -L /path/to/cross/boost/lib/ -lboost_system -pthread
EDIT 2:
When linked with
rpi-g++ -o myprog *.o -L /path/to/cross/boost/lib/ -rdynamic -lboost_system -pthread
the problem remains the same. I have checked and verified everything suggested by Technaton as well. Strangely, ldd insists that the created executable is "not a dynamic executable" (checked that on my PC and on the RPi), which doesn't make sense to me.
There are several things you can check. I've posted a complete check list here, but judging from your linker command line, number 5 is probably the culprit.
Check that your library and your program are correctly build for the target architecture. You can verify that by using file ./myprog and file libboost_system.so.1.60.0.
Make sure that you have copied the actual shared object, and not a link to it.
Ensure that the shared object file's permissions are sane (0755).
Run ldconfig -v and check that your shared object file is picked up. Normally, /usr/local/lib is in the standard library search path, and LD_LIBRARY_PATH is not required.
Make sure that your program is actually dynamically linked by running ldd ./myprog. Judging from your linker command line, that is the problem: You're missing -rdynamic.
Check the paths returned from ldd: If you have linked with rpath, the library search path might be screwed up. Try again without -rpath.
I'm trying to create a shared object (.so) that will make it so, by including one shared object with -lboost, I implicitly include all the boost libraries. Here's what I tried:
#!/bin/sh
BOOST_LIBS="-lboost_date_time-gcc43-mt -lboost_filesystem-gcc43-mt"
#truncated for brevity
g++ $BOOST_LIBS -shared -Wl,-soname,libboost.so.1 -o libboost.so.1.0
ln -si libboost.so.1.0 libboost.so.1
ln -si libboost.so.1 libboost.so
After placing all 3 created files (libboost.so libboost.so.1 libboost.so.1.0) in the same directory as all the boost libraries, I tried compiling a test program with it (which depends on -lboost_date_time-gcc43-mt):
g++ -lboost test.cpp
Doing this, I got the same undefined reference message as not having -lboost. Having -lboost_date_time-gcc43-mt works, but that's too wordy :) How do I get -lboost to automatically bring in the other shared libraries?
You don't. Not really, anyway.
The linker is stripping out all of the symbol dependencies because the .so doesn't use them.
You can get around this, perhaps, by writing a linker script that declares all of the symbols you need as EXTERN() dependencies. But this implies that you'll need to list all of the mangled names for the symbols you need. Not at all worth the effort, IMO.
I don't have a solution for creating a dummy '.so', but I do have something that will simplify your life... I highly suggest that you try using cross-platform make (CMake). In CMake, linking against those libraries is easy:
FIND_PACKAGE(Boost 1.37 COMPONENTS date_time filesystem REQUIRED)
ADD_EXECUTABLE(myexecutable ${myexecutable_SRCS})
TARGET_LINK_LIBRARIES(myexecutable ${Boost_LIBRARIES})
The commands above, if placed in a "CMakeLists.txt" file, is all you need to:
Verify that Boost 1.37 or later is installed, with the "date_time" and "filesystem" libraries installed.
Create an executable named "myexecutable" from the sources listed in the corresponding variable.
Link the executable "myexecutable" against the boost "date_time" and "filesystem" libraries.
See also: Why the KDE project switched to CMake.
Actually, making one .so depend on all boost .so files is quite possible (but might not actually help you). I've just tried this:
$ export BOOST_ROOT=/home/ghost/Work/Boost/boost-svn
$ g++ -shared -Wl,-soname,libboost.so -o libboost.so $BOOST_ROOT/stage/lib/libboost_program_options.so
$ g++ -L . -I $BOOST_ROOT first.cpp -lboost -Wl,-R$BOOST_ROOT/stage/lib
$ LD_LIBRARY_PATH=.:$BOOST_ROOT/stage/lib ./a.out
And it did work. However, note that dancing with -R and LD_LIBRARY_PATH. I don't know an way how you can include the path to Boost .so inside your libboost.so so that they are used both for linking and actually running the application. I can include rpath inside libboost.so just fine, but it's ignored when resolving symbols for the application.