I have built a cross-platform C++ application that uses some external libraries (e.g. Boost, GLFW). My intention is to create an archive, for instance a .zip file, for each operating system so that the end user only has to extract this package in order to run the application. I am facing a problem related to dynamic linking of the libraries.
When I built the application in Linux, I use the -Wl,rpath option to specify the run time library search path. On Windows, I can simply place all relevant .dll files in the same folder as where the executable resides. Now on MacOS, I am facing the issue that the libraries are referred to by absolute paths. Those paths may be valid for my system, but not necessarily for the system of the user. I already found that I can change those paths to relative paths using install_name_tool.
The remaining difficulty is that I have to change quite a lot of these references. Besides the references in the executable, some of the libraries have dependencies themselves which are also defined by absolute paths.
My question is: Is there an efficient and smart way to set relative paths instead of absolute paths for the libraries on MacOS?
Related
I have an issue where I'm developing on one system where Boost is installed on:
/usr/include
/usr/lib
On a system I will deploy this on, the libboost libraries are at:
/nfs/mount/boost
/nfs/mount/lib
And I can't go about changing every system I deploy on to install libboost in the same place.
Is there a way to either:
include libboost as part of the binary executable such that loading from the system lib paths is not needed.
make the executable search for different directories when trying to load to libboost?
I'm using g++ 8
Sounds like you need a more sophisticated build environment.
I'm not sure what you mean here:
include libboost as part of the binary executable such that linking is not needed
Linking is not something you can skip. If you are trying to avoid distributing .dll/.so files with your executable, you need to avoid using the portions of the boost library that require compilation of the boost binaries (i.e. those not listed here https://www.boost.org/doc/libs/1_62_0/more/getting_started/windows.html#header-only-libraries).
Of course, that is not often the case. So...
make the executable search for different directories when trying to link to libboost?
This will never work reliably as you scale and is a nightmare in the CI world. This is where package managers such as conan (https://conan.io/) come to save the day. Delegating the package management to a third-party is the most reliable way of getting your code to build across multiple environments/platforms.
Also, building your executable and distributing it are separate operations. Any dynamically linked libraries will need to be discoverable on the system path at runtime.
Probably a newbie's question for those used to package and distribute applications:
when launching my executable on a different computer than the one used for compilation, I need to place several dynamic libraries in the same folder to run it.
I want the executable to look at runtime in a relative lib/ subfolder placed besides it and where I would have placed all these libraries. What do I have to do for this?
I don't know about Win and MacOS, but for Linux, the good solution would be to set the RPATH binary header of the executable during build-time.
For me this article helped a lot to understand the various ways dynamic libraries are looked up.
How to set the RPATH properly depends on your build system, but basically you need to specify the option -rpath=/path/to/lib/folder for the linker - this will have the effect that the specified path will always be searched first for shared libraries.
To check if your program was linked correctly, you can use
readelf --dynamic [executable or shared library] | grep PATH
It would be also possible to use the LD_LIBRARY_PATH environment variable, or edit /etc/ld.so.conf to set the path where the libraries are located, but these would require changes on the system where the program is run.
In the project I am working on, we provide the possibility to dynamically load additional features. For that we use dlopen.
To find this libraries we have something we call module path. There we have a default path, where the shared libraries are (a lot of them are shipped).
At the moment we have two default paths: we first look in the build directory for the shared library and afterwards in the install directory. This is because it should also be possible to run the application without installing it (so in that case it needs to look first in the build directory).
Now the problem ist, if a user builds the application from source and installs it with make install, the libraries in her build directory are loaded by default. This will result in a crash. So it only works if the user afterwards removes or renames the build directory.
No the question: is there a trick (either by C++ or by the build system) to know whether the application is installed or not. The problem is, that the functionality is implemented in a shared library and the implemented way to search for modules has to work also for other applications that link against our library (so we can not rely on the path of the executable). We use CMake as a build system.
To make the situation even harder, the solution has to work on Windows, Linux and Mac OS X.
EDIT:
I further investigated and the problem is more complicated. This is the situation:
There is a small executable a
Furthermore there is a "main" library main.so
then there is a dynamically loaded library lib.so
lib.so links against main.so
The problem is, that lib.so has the absolute path to main.so in the build directory in its rpath. Thanks to the tip of #MSalters I now was able to make a hack to make sure to load the correct version of lib.so (the one in the install directory) but since it has the build path in the rpath it loads the wrong main.so (so in fact there are two copies of main.so in the memory - this messes things up).
Is there a way to remove this reference to the build path from the library? I tried all options of cmake related to rpath without success
Can't you check where the executable itself is? If it's in the build directories, use build libraries -- if it's in the install, use install?
getcwd() has equivalents on all of those platforms, but it might not be what you want -- it depends on how you run the executable.
To get the process's location is system specific, I think, but it shouldn't be too hard to wrap that.
The installed version should not have the build directory in the rpath.
You might want to do the linking twice (once for the build version and once for the installed version). Usually, on *nix systems, the installed binary has some static path where it tries to find plugins. You might define some environment variable (or command-line argument) to overload it for the build execution (and use a wrapper script to set it in the build environment).
Check how it is solved by some projects (Firefox for example).
I don't know much about windows system but I think the standard way of doing this is to search plugins in the same directory as the executable.
I pulled out an application that I wrote in C++ using the sfml library, but I'm having trouble setting up the library in Eclipse. I specified the include path, the lib path and included all the necessary .so libraries to link to. the application compiles fine but it complains at runtime about missing libraries. Why is this happening? Didn't I include the path to the libraries in the project settings already? I have even tried to place all the .so's in the executable directory with no luck.
There is only the name of the shared lib stored in the executable. At program startup the dynamic linker then searches for the specified libs in its search paths. You can add/specify search paths by placing them colon separated in the environment variable LD_LIBRARY_PATH or by specifying them in /etc/ld.so.conf (at least if you use some unix based OS). On windows the whole PATH environment variable is used when searching for dynamic-link libraries (DLL).
To see the paths of shared libraries used by a given application run ldd applicationPath.
I am trying to get my head around the way shared libraries work in the c++ unix environment. I understand we only need header files and no shared libraries specification when compiling code. But if I want to create an executable or shared library from my compiled files, do I need to specify shared library dependencies (those are dynamic)? And do the paths of shared libraries need to match the path at runtime loading?
I am using Linux 2.6.18-164.11.1.el5 #1 SMP x86_64 GNU/Linux
I am having a problem where my code is not able to pick up a library at runtime. I have tried setting LD_LIBRARY_PATH and PATH. But at runtime when I run the executable, I get the following error:
Error: librc.so: cannot open shared object file: No such file or directory
Sam
The headers are only for the compile phase. At link time, you usually have to specify which shared libs you are going to link to. You might see -L options to set locations to where shared libraries reside, and/or -l to specify which libraries to link. There is usually also a switch on the command line to alert the linker as to whether you are using thread-safe versions of the libs or the 'regular' ones, and another switch to specify dynamic linking.
At run time, whether you are starting the program that uses the libs, or running ldd to find out what it needs, the OS has a system for locating .so files, and this can vary from one unix version to another. The LD_LIBRARY_PATH variable specifies where to look for .so files, but may not be the full story, depending on the exact unix version in question. Also, you probably don't want to fiddle around with modifying LD_LIBRARY_PATH except from a throw-away shell, since it has system wide effects. A better option is to check it the 'missing' .so files are or are not on the existing path set by LD_LIBRARY_PATH, and if not, try putting copies of them somewhere on that path.
At run time, dynamic libraries are searched:
in a path recorded in the executable (under linux with -rpath at link time, under Solaris with -R, using $ORIGIN in a directory name allows to specify a directory relative to the directory containing the executable)
in the LD_LIBRARY_PATH (or equivalent, there are sometimes 64/32 bits variant). If a path has been recorded in the executable, LD_LIBRARY_PATH may not searched (under Linux it is searched after the recorded path if the executable has been linked with the option --enable-new-dtags; I don't remember Solaris behavior for now)
in a set of system dependant directories (Linux allows to specify them in /etc/ld.so.conf and has a cache, see ldconfig)