I have a folder where many Shared Libraries are in. I want to load all of them in my program so iterate them and load them but some got not loaded because it could not find a dependency which lies in the same folder but (at least in windows) it doesn't search the same folder.
I googled and found 'AddDllDirectory' for windows.
Is there a equivalent for this function in Linux and Mac?
If not how could I workaround this problem?
It needs to be runtime because the folder containing the Shared Libraries can change.
For a single, specific program, execute it from the shell (or with a script) like this:
LD_LIBRARY_PATH=path:to:libraries /path/to/program
To set up system wide libraries, adjust the contents of /etc/ld.so.conf.d or /etc/ld.so.conf.
If you just want to make your programs refer to a specific additional path, have a look at the rpath linker flags. E.g.
gcc -o my_program -Wl,-rpath='additional/library/paths'
With rpath take note about the special "variable" ${ORIGIN} which allows you to specify paths relative to the install location of the linked binary. For details read the manual page of ld.so
Related
I have a question related to how to install a built executable program with cmake when it relies on some external libraries. Suppose my executable is abc, and it relies on two external libraries: lib1.so and lib2.so. The structure of the codes are as follows:
-.........
|----bin (lib1.so lib2.so)
|----include(lib1.h lib2.h)
|----src(main.cpp)
When the executable program is installed using the following cmake commands:
INSTALL(TARGETS ${Exe_Name}
RUNTIME DESTINATION Path to bin
LIBRARY DESTINATION Path to bin)
I expect that the executable program will be in the same directory with lib1.so and lib2.so. However, when I execute the built program in the installation folder, I met the following error:
error while loading shared libraries: lib1 can not open shared object file No such file or directory
If I use ldd to check the executable, I found lib1.so and lib2.so not found. After searching for possible solutions, I found if I call the executable in this way, then it worked:
LD_LIBRARY_PATH=./ ./my_program_run
Then my question is how I can let my executable program knows the locations of the shared libraries with cmake when it is installed? Thanks.
This is best solved this with the RPATH of the final executable. RPATH is a hardcoded search path for the executable itself, and allows the use of the string $ORIGIN, which expands to the location of the executable at runtime. See this reference: http://man7.org/linux/man-pages/man8/ld.so.8.html
CMake strips the rpath of a binary at installation time, to avoid the binary picking up libraries littered around your development tree. But it also provides a simple way to modify the installation rpath for exactly this reason. Here's the short answer:
IF(UNIX)
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_RPATH}:\$ORIGIN/../bin:\$ORIGIN")
ENDIF()
This particular example appends to the existing rpath, and adds . and ../bin to the search path, all relative to the location of the binary.
Some developers claim that adjusting the RPATH of the binary is not a good idea. In the ideal world, all the libraries would live in the system library directories. But if you take this to the extreme, you end up with Windows (at least the older ones), where c:\windows\system32 is full of junk that came from who knows where, and may or may not conflict with other software, etc. Using rpath and installing everything in one place seems like a great solution.
If the application is to be cleanly installed to a standard linux distribution, then you should either install the supporting shared libraries into a standard location (/usr/lib), or you should add the libraries location to the ld.so config, by create an /etc/ld.so.conf.d/myprogram.conf file containing the name of the directory the libraries are in.
If the installation is temporary or more ad-hoc, then a script to set the LD_LIBRARY_PATH is suitable.
The libraries are searched in the predefined locations which includes standard library paths configured with ld.so.conf and LD_LIBRARY_PATH. You can also try to compile your app with -rpath, but it is not recommended by some developers. I suggest you to create a wrapper script which will set LD_LIBRARY_PATH and run the real application like that:
"theapp" script:
#!/bin/sh
dir="`dirname \"$0\"`"
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}"$dir"
exec "$dir/theapp.real" # your real application
The application, script and libraries should be in the same location (under bin/).
Is it possible to compile a program with g++ so that shared libraries etc are "included" with the executable?
I have a c++ program that I'd like to compile and run at another location where I'm missing some libraries and don't have install access.
The main reason I couldn't find answers for this is probably that I don't know how to call it..
No, it's not possible.
Either link statically (with -static) so it doesn't use any shared libraries, or copy the shared libraries to the other location along with the executable.
Since the shared libraries will not be in the dynamic loader's usual search paths you'll need to ensure they can be found, either by setting the LD_LIBRARY_PATH environment variable when running the program, or by setting an RPATH in the executable when you build it.
(Assuming you're using the GNU linker ...) To set an RPATH in the executable link with '-Wl,-rpath,$ORIGIN' (the quotes are important, to stop $ORIGIN being expanded by the shell). That means the loader will look for shared libraries in the same directory as the executable itself.
See https://gcc.gnu.org/onlinedocs/libstdc++/faq.html#faq.how_to_set_paths and https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dynamic_or_shared.html#manual.intro.using.linkage.dynamic for more information.
Probably a newbie's question for those used to package and distribute applications:
when launching my executable on a different computer than the one used for compilation, I need to place several dynamic libraries in the same folder to run it.
I want the executable to look at runtime in a relative lib/ subfolder placed besides it and where I would have placed all these libraries. What do I have to do for this?
I don't know about Win and MacOS, but for Linux, the good solution would be to set the RPATH binary header of the executable during build-time.
For me this article helped a lot to understand the various ways dynamic libraries are looked up.
How to set the RPATH properly depends on your build system, but basically you need to specify the option -rpath=/path/to/lib/folder for the linker - this will have the effect that the specified path will always be searched first for shared libraries.
To check if your program was linked correctly, you can use
readelf --dynamic [executable or shared library] | grep PATH
It would be also possible to use the LD_LIBRARY_PATH environment variable, or edit /etc/ld.so.conf to set the path where the libraries are located, but these would require changes on the system where the program is run.
In the project I am working on, we provide the possibility to dynamically load additional features. For that we use dlopen.
To find this libraries we have something we call module path. There we have a default path, where the shared libraries are (a lot of them are shipped).
At the moment we have two default paths: we first look in the build directory for the shared library and afterwards in the install directory. This is because it should also be possible to run the application without installing it (so in that case it needs to look first in the build directory).
Now the problem ist, if a user builds the application from source and installs it with make install, the libraries in her build directory are loaded by default. This will result in a crash. So it only works if the user afterwards removes or renames the build directory.
No the question: is there a trick (either by C++ or by the build system) to know whether the application is installed or not. The problem is, that the functionality is implemented in a shared library and the implemented way to search for modules has to work also for other applications that link against our library (so we can not rely on the path of the executable). We use CMake as a build system.
To make the situation even harder, the solution has to work on Windows, Linux and Mac OS X.
EDIT:
I further investigated and the problem is more complicated. This is the situation:
There is a small executable a
Furthermore there is a "main" library main.so
then there is a dynamically loaded library lib.so
lib.so links against main.so
The problem is, that lib.so has the absolute path to main.so in the build directory in its rpath. Thanks to the tip of #MSalters I now was able to make a hack to make sure to load the correct version of lib.so (the one in the install directory) but since it has the build path in the rpath it loads the wrong main.so (so in fact there are two copies of main.so in the memory - this messes things up).
Is there a way to remove this reference to the build path from the library? I tried all options of cmake related to rpath without success
Can't you check where the executable itself is? If it's in the build directories, use build libraries -- if it's in the install, use install?
getcwd() has equivalents on all of those platforms, but it might not be what you want -- it depends on how you run the executable.
To get the process's location is system specific, I think, but it shouldn't be too hard to wrap that.
The installed version should not have the build directory in the rpath.
You might want to do the linking twice (once for the build version and once for the installed version). Usually, on *nix systems, the installed binary has some static path where it tries to find plugins. You might define some environment variable (or command-line argument) to overload it for the build execution (and use a wrapper script to set it in the build environment).
Check how it is solved by some projects (Firefox for example).
I don't know much about windows system but I think the standard way of doing this is to search plugins in the same directory as the executable.
I am trying to get my head around the way shared libraries work in the c++ unix environment. I understand we only need header files and no shared libraries specification when compiling code. But if I want to create an executable or shared library from my compiled files, do I need to specify shared library dependencies (those are dynamic)? And do the paths of shared libraries need to match the path at runtime loading?
I am using Linux 2.6.18-164.11.1.el5 #1 SMP x86_64 GNU/Linux
I am having a problem where my code is not able to pick up a library at runtime. I have tried setting LD_LIBRARY_PATH and PATH. But at runtime when I run the executable, I get the following error:
Error: librc.so: cannot open shared object file: No such file or directory
Sam
The headers are only for the compile phase. At link time, you usually have to specify which shared libs you are going to link to. You might see -L options to set locations to where shared libraries reside, and/or -l to specify which libraries to link. There is usually also a switch on the command line to alert the linker as to whether you are using thread-safe versions of the libs or the 'regular' ones, and another switch to specify dynamic linking.
At run time, whether you are starting the program that uses the libs, or running ldd to find out what it needs, the OS has a system for locating .so files, and this can vary from one unix version to another. The LD_LIBRARY_PATH variable specifies where to look for .so files, but may not be the full story, depending on the exact unix version in question. Also, you probably don't want to fiddle around with modifying LD_LIBRARY_PATH except from a throw-away shell, since it has system wide effects. A better option is to check it the 'missing' .so files are or are not on the existing path set by LD_LIBRARY_PATH, and if not, try putting copies of them somewhere on that path.
At run time, dynamic libraries are searched:
in a path recorded in the executable (under linux with -rpath at link time, under Solaris with -R, using $ORIGIN in a directory name allows to specify a directory relative to the directory containing the executable)
in the LD_LIBRARY_PATH (or equivalent, there are sometimes 64/32 bits variant). If a path has been recorded in the executable, LD_LIBRARY_PATH may not searched (under Linux it is searched after the recorded path if the executable has been linked with the option --enable-new-dtags; I don't remember Solaris behavior for now)
in a set of system dependant directories (Linux allows to specify them in /etc/ld.so.conf and has a cache, see ldconfig)