C++ linux: dlopen can't find .so library - c++

Reworded Question (although it's been solved already):
I've been having trouble using dlopen(3) to load a shared object library on linux. The library is part of a system of libraries built by me that are all loaded at runtime by a central executable. All of this is organized into a single workspace in Code::Blocks, where each project is given its own folder within a directory called Source, which is to be shipped with the program. The build directory of the executable is two directories backward from its own source code so that the exectuable and the Source folder are in the same directory, The libraries also build to the same directory as the executable, so naturally I pass the name of the library I'm trying to open as shown:
int main(int argc, char** argv) {
void* hLibrary = dlopen("libLibrary.so", RTLD_NOW | RTLD_GLOBAL);
if(hLibrary == NULL) {
fprintf(stderr, "%s\n", dlerror());
return 1;
}
return 0;
}
This was working at one point when the build directory was the same as the source code, until I changed the directories of the source code around to the arrangement described above. The problem at this point is that dlerror() returns "Cannot open libLibrary.so: no such file or directory," even though the file clearly exists and is in the same directory as the executable. I then tried passing in "/libLibrary.so" instead, because according to the man page on dlopen(3), adding a / indicates a relative directory. This returned the same error.
The solution to this was that a "./" was needed - where "." represents the working directory of the executable - and the working directory needed to be changed in Code::Blocks to where the executable was to be built. The following works perfectly:
void* hLibrary = dlopen("./libLibrary.so", RTLD_NOW | RTLD_GLOBAL);
This doesn't really show the full solution, but the following is basically the equivalent of what I'm doing:
void* hLibrary = dlopen("./../../libLibrary.so", RTLD_NOW | RTLD_GLOBAL);
Hopefully this explains the situation a little better.

Read the dlopen(3) man page (e.g. by typing man dlopen in a terminal on your machine):
If filename contains a slash ("/"), then it
is interpreted as a (relative or absolute) pathname. Otherwise, the
dynamic linker searches for the library as follows (see ld.so(8) for
further details):
o (ELF only) If the executable file for the calling program
contains a DT_RPATH tag, and does not contain a DT_RUNPATH tag,
then the directories listed in the DT_RPATH tag are searched.
o If, at the time that the program was started, the environment
variable LD_LIBRARY_PATH was defined to contain a colon-separated
list of directories, then these are searched. (As a security
measure this variable is ignored for set-user-ID and set-group-ID
programs.)
o (ELF only) If the executable file for the calling program
contains a DT_RUNPATH tag, then the directories listed in that
tag are searched.
o The cache file /etc/ld.so.cache (maintained by ldconfig(8)) is
checked to see whether it contains an entry for filename.
o The directories /lib and /usr/lib are searched (in that order).
So you need to call dlopen("./libLibraryName.so", RTLD_NOW) -not just dlopen("libLibraryName.so", RTLD_NOW) which wants your plugin to be in your $LD_LIBRARY_PATH on in /usr/lib/ etc .... - or add . to your LD_LIBRARY_PATH (which I don't recommend for security reasons).
As Jhonnash answered you should use and display the result of dlerror when dlopen (or dlsym) fails:
void* dlh = dlopen("./libLibraryName.so", RTLD_NOW);
if (!dlh)
{ fprintf(stderr, "dlopen failed: %s\n", dlerror());
exit(EXIT_FAILURE); };
You might want to read some books like Advanced Linux Programming to get some knowledge about Linux system programming in general.

About the dlopen is defined; Dynamic library dlopen error can be checked. Undefined symbol error by this link.

Related

Is it possible to get location of dependent libraries from within code

I want to load some dependent libraries from my program with dlopen function. Is it possible to know actual location of these libraries?
eg, ldd shows all dependent libraries with paths in system. How does it work? Is it possible to get paths to correspondent libraries I need to load with dlopen via some call from my C++ code?
From man dlopen one can read:
The function dlopen() loads the dynamic library file named by the null-terminated string filename and returns an opaque "handle" for the dynamic library. If filename is NULL, then the returned handle is for the main program. If filename contains a slash ("/"), then it is interpreted as a (relative or absolute) pathname. Otherwise, the dynamic linker searches for the library as follows (see ld.so(8) for further details):
(ELF only) If the executable file for the calling program contains a DT_RPATH tag, and does not contain a DT_RUNPATH tag, then the directories listed in the DT_RPATH tag are searched.
If, at the time that the program was started, the environment variable LD_LIBRARY_PATH was defined to contain a colon-separated list of directories, then these are searched. (As a security measure this variable is ignored for set-user-ID and set-group-ID programs.)
(ELF only) If the executable file for the calling program contains a DT_RUNPATH tag, then the directories listed in that tag are searched.
The cache file /etc/ld.so.cache (maintained by ldconfig(8)) is checked to see whether it contains an entry for filename.
The directories /lib and /usr/lib are searched (in that order).
So, if the desired library is "installed", a simple dlopen("foobar.so", flag) will do.

Using -L and -l flags vs giving library file as input

What's the difference between writing:
g++ test.cc -L/my/dir/ -lname
and
g++ test.cc /my/dir/libname.so ?
Are both correct?
The things that I can think of:
First one is cross platform, g++ from MINGW will look for DLL's
In the second one we explicitly specify we want dynamic linking
The difference is that
g++ test.cc -L/my/dir/ -lname
may succeed even if /my/dir/libname.so does not exist:
It can find libname.so from a system library directory (usually /lib, /usr/lib).
It can find libname.a, and do static linking.
Furthermore, it allows you to link to multiple libraries from the same directory without repeating the parent directory.
The first method looks into /my/dir directory for the library file at link time. However only a short library name libname.so (or whatever it finds according to OS) is written into the image. When the program starts, it will look for the library in some library path, i.e. LD_LIBRARY_PATH on linux. Now you can relocate the executable and the library into different new places and set a correct search path after relocation.
The second method writes the full path of the library into the image: /my/dir/libname.so. When the program starts, it will look for this particular path to get the library. This makes the image non-relocatable, unless the same library path exists in the new environment. The effect is similar to the '-rpath' method.

C++ linux executable keeps trying to use library that does not exist

I am trying to write a simple application with GLFW on Linux. Right now the main file (the only file) is basically just a few lines of code to make sure the dynamic library linked correctly. Here it is:
#include <GLFW/glfw3.h>
#include <iostream>
int main()
{
glfwInit();
std::cout << "It works this far!" << std::endl;
glfwTerminate();
}
The include files are stored in a directory labelled "include" and the library files are stored in a directory labelled "lib". As of right now, I am compiling the program with the following line:
g++ -Wl,-Rlib -Iinclude -Llib test.cpp -o test -lglfw.3.2
It compiles and links just fine, but when I try to execute it, I get the following error:
./test: error while loading shared libraries: libglfw.so.3: cannot open shared object file: No such file or directory
Now, before you rush to downvote this question into oblivion and mark it as a duplicate, at least allow me to explain why I believe my question is different enough to not be a duplicate. I already attempted the solutions that the other questions presented, but it was unsuccessful. As you can see, I tried setting the path to the library during linking with the -Wl,-Rlib tag. I also tried setting LD_LIBRARY_PATH to point to the location of my libraries (the 'lib' folder), but it still threw the same error. (It didn't matter if the path was relative or absolute.)
So, the next thing I tried was running the ldd command on the executable. I got some other dependencies that were working just fine, but importantly, I got this:
libglfw.so.3 => not found
For some reason, it insists on looking for libglfw.so.3. It will not have it any other way. Upon renaming the library from libglfw.3.2.so to libglfw.so.3, the program executed just fine and printed It works this far! as if there were no problems at all.
Why would this happen?
For some reason, it insists on looking for libglfw.so.3. ... Upon renaming the library from libglfw.3.2.so to libglfw.so.3 ...
The ELF executables contain the exact name of the dynamic libraries used.
If the executable contains the library name "libglfw.so.3" the file must be named exactly like this.
The file naming scheme is intentionally done in a way that not the "full" version is coded into the file name: This way a later version ("libglfw.so.3.15") will work with the executable.
Normally there should be a symbolic link to the latest version of the library installed:
libglfw.so.3 -> libglfw.so.3.2
This symbolic link seems to be missing on your computer. I would say that this is an installation problem!
EDIT
The question could be: Why is the file name stored in the executable file not libglfw.3.2.so but libglfw.so.3?
The answer has to do with the backward compatibility when a new version of a library is installed:
Normally you would use the switch -lglfw and a symbolic link named libglfw.so is looked up.
If you stored the file name libglfw.so in the executable file and a new, incompatible version if this library (libglfw.so.4) is installed you would have no chance to get the program running by having both versions of the library installed.
To enable backward compatibility by having both versions of the library installed the "real" symbolic link name of the library (libglfw.so.3) must be stored in the executable file.
Therefore the "expected" file name of a library is stored in the library itself: Inside the file libglfw.so.3.2 you'll find some information that the file expects itself to be stored as libglfw.so.3.
The linker will use this information about the file name because it assumes that the library name given in the linker switch (-lglfw) is less "precise" than the name stored in the library itself.
For some reason, it insists on looking for libglfw.so.3. It will not have it any other way.
This is the Linux convention for shared libraries which is described here among other places. For Linux libfoo.so.x.y.z is considered to have the same ABI as libfoo.so.x. Usually when shared libraries are installed (e.g. via rpm, dpkg, etc.) there's an invocation of ldconfig that happens so that the just installed libraries have a symlink following the convention installed that references the library. Also these libs (if installed to a "trusted location"), are added to a linker cache for performance reasons.
It compiles and links just fine, but when I try to execute it, I get the following error:
./test: error while loading shared libraries: libglfw.so.3: cannot open shared object file: No such file or directory
libglfw.so.3 isn't on ld-linux.so's path.
As you can see, I tried setting the path to the library during linking with the -Wl,-Rlib
Still won't find it -- libglfw.so.3 isn't on ld-linux.so's path. You can add it by doing something like:
ldconfig -n /path/to/lib
Which should output the requisite libglfw.so.3 symlink for your lib.
IIRC setting the rpath might require a full path.
I also tried setting LD_LIBRARY_PATH to point to the location of my libraries
Again, libglfw.so.3 isn't on ld-linux.so's path.

"cannot open shared object file," but it exists

I'm having trouble finding why this library (matio) isn't working for me. In my Makefile I have this:
LIBS += -L/home/brian/.../matio-1.5.6/src/.libs/ -lmatio
When I attempt to run my code (links fine) I get this error:
error while loading shared libraries: libmatio.so.4: cannot open shared object file: No such file or directory
libmatio.so.4 exists in the directory specified by the -L flag.
I built the library and it seems to go through make check with only a handful writing errors (which is fine as I only need it for reading).
Things I've tried:
Specifying the name (i.e. -l:libmatio.so.4.0.2)
Adding the path to LD_LIBRARY_PATH
Adding the path as a line in /etc/ld.so.conf and run sudo ldconfig
Adding a new file in /etc/ld.so.conf.d with the path and run sudo ldconfig
(When I run ldconfig -p | grep matio nothing returns. Am I doing something wrong with ldconfig?)
The error is actually telling you "no compatible library with that name exists in the library cache", not "no file with that filename exists on disk".
So, confusingly, this can happen when the shared object file is in the wrong format.
Ensure that it was built for the right platform by the right compiler! You can have a look with file and verify that the dynamic link is failing using ldd on your executable.

Setting a System Wide path in Mac Yosemite

I have installed boost libraries. I am using Eclipse to make a simple boost project.
#include <stdio.h>
#include <boost/filesystem.hpp>
int main() {
boost::filesystem::path path("/Users/schoen"); // random pathname
bool result = boost::filesystem::is_directory(path);
printf("Path is a directory : %d\n", result);
return 0;
}
I have already set the path of include folder and library folder in the properties of this project. But I was getting a runtime error: dyld: Library not loaded: libboost_system.dylib. To solve this problem, I went to run configuration and set the environment variable DYLD_LIBRARY_PATH to /Users/myName/Documents/Softwares/boost_1_59_0/stage/lib. This has solved my problem.
What I need: I don't want to set the environment variable for each Boost-based project. Therefore, I tried to set my .bash_profile. I wrote the following lines in .bash_profile.
# Following lines are for Boost Library
DYLD_LIBRARY_PATH=/Users/myName/Documents/Softwares/boost_1_59_0/stage/lib
export DYLD_LIBRARY_PATH
PROBLEM: By setting the variable in .bash_profile, I am able to run my program through the terminal. The program also runs if I open the IDE (e.g. Eclipse) via terminal and then run the application. Apparently, .bash_profile can set the variable for terminal applications. How can I set the path for windowed applications too?
PS: This path setting problem is not just for Boost only, but I have to do similar things for other libraries too (such as OpenCV which is build/installed in a local directory).
Rather than adding a system wide path like this, you should add the rpath to the libraries to applications that depend on boost. To add the rpath option, you do Project Properties -> C/C++ Build -> Settings -> Miscellaneous and in the linker flags add:
-Wl,-rpath,/Users/myName/Documents/Softwares/boost_1_59_0/stage/lib
(this is if your linker is g++ or clang++, for example)
If your linker is ld explicitly, then the option is
-rpath /Users/myName/Documents/Softwares/boost_1_59_0/stage/lib
although you may need to add -macos_version_min 10.5 (or newer - probably 10.8 - this depends on the OS you're building on).
This will cause any applications built to search there for libraries as well as the default locations.
Although a location like that to me looks a bit volatile
Turns out that boost builds without setting the library name to include #rpath in the install name, which means that even when you set the -rpath in the build, because the libraries aren't mentioned to be in the #rpath, it won't find them at run time. A workaround to this is to explicitly set the install name for the boost libraries, and their internal references to their own libraries:
#!/bin/bash -p
for i in *.dylib; do
# set the rpath
install_name_tool -id #rpath/$i $i
for lib in $(otool -L $i | grep libboost | awk '{print $1}'); do
if [[ -f $lib ]]; then
install_name_tool -change $lib #rpath/$lib $i
fi
done
done
This means the binaries that are linked to these boost libraries will respect the rpath setting.
You can repeat a similar process for other libraries to ensure that they respect rpath. The key element is the install_name_tool -id "#rpath/libstuff.dylib" libstuff.dylib, which says that when you link to the library record a reference to #rpath/libstuff.dylib. If libraries don't already have this set.
Secondly, for internal references to dependent libraries, the -change option alters references to the absolute name to an rpath relative name e.g. install_name_tool -change "libstuff.dylib" "#rpath/libstuff.dylib" libdependsonlibstuff.dylib. This can even be performed on a linked binary.
If you still want to set an environment variable, there are some options available which should help you getting a solution which works in that case.