For various reasons mostly to do with inertia, we don't have a make install target.
Rather, we build our large C++ codebase directly into an FHS-like tree;
output/
bin/
lib/
etc/
...
We've recently switched some third-party libraries to dynamic linking, and so we push a number of .so libraries into lib/.
Now, we're used to being able to just launch our executables from bin/, but that no longer works because the loader doesn't search our lib/ directory.
LD_LIBRARY_PATH would solve this, but we would prefer not to have to provide it before every single executable invocation, and we don't want to stick it in the shell's environment, because we typically switch between a number of different build trees in the same shell.
We've considered adding an rpath entry in the generated ELF, but relative paths are typically resolved against $PWD, not the executable's dirname.
Is there a way to nudge the loader to look in dirname(argv[0])/../lib for .so libs?
Basically, I understand that there are lots of ways we can change our habits to make this work (and probably should), but we prefer not to at this point, so can we coerce the Linux so loader to do what we want? Thanks!
Yes, it is possible using rpath and ${ORIGIN} macro, which is recognized by ld.so at runtime.
From man ld.so:
ld.so understands certain strings in an rpath specification
(DT_RPATH or DT_RUNPATH); those strings are substituted as follows
$ORIGIN (or equivalently ${ORIGIN})
This expands to the directory containing the application executable.
More variables are available. You don't need to coerce the loader to anything. It has the feature for you. :)
Related
We're using Jenkins 2.60.2 and CMake 3.9.1 to automate our build system. This all works well for multiple versions of build tools, architectures and debug/release targets (if ALL configurations have been built and installed, so both Debug AND Release).
A Debug-only configuration that uses find_package() typically ignores the CMAKE_BUILD_TYPE at discovery. Internally the scripts search for file and libraries and store the locations in variables. At the end of the script, the variables are scanned for _NOTFOUND strings, which is the result of a file or library not found in all the reference paths/hints. So essentially a find_package() will fail if the Release lib can not be found, and mark the whole package as not installed properly, even though the build is only strictly interested in the Debug target.
Typically the XXXConfig.cmake files use a call to find_package_handle_standard_args(.. PATH_TO_LIB) that scans for _NOTFOUND strings in the path variables to the libraries. These variables typically get set to _NOTFOUND by earlier calls to find_library(PATH_TO_LIB libname ..). For more information I refer to the CMake docs.
The user can indeed tag debug libraries with 'debug' and release libs with 'optimized', but this does not seem to help during the lib discovery and is only used during linking.
Anyone knows how to handle this properly?
Kind regards
This is one of the unfortunate shortcomings of the classic use of find_package.
Note that find_package also allows a different mode of operation, based on config file packages, which is well-suited to address this particular problem, but will require some changes to your build system. You will need config scripts for all your libraries (CMake can generate them for you if the libraries are themselves also built by CMake; if not, this can be a bit of a hassle), and depending targets will refer to those libraries via imported targets instead of variables (which usually makes things way easier for those depending targets). I would strongly recommend you adopt this as the long-term solution.
If for some reason you cannot do this, you will have to modify your find scripts. A common technique is to search for debug and release binaries separately, but then combine the find libraries from those calls into a single variable (together with the debug and optimized specifiers) and then have that variable as an argument to find_package_handle_standard_args. That way, as long as one of the two is found, your find script will be happy, although you might not be able to build all possible configurations in the end. Alternatively, you can also skip the call to find_package_handle_standard_args altogether and manually implement your own logic for detecting whether the library was found. As you can see from the manpage for that function, it does mostly boilerplate stuff and can be easily replaced by a more flexible, handwritten implementation if necessary.
I have a system-wide libc++.so in /usr/lib64. I want to link my binary against another libc++.so which is located somewhere else, say, in $HOME/.local/lib. Also, I want to be able to find all other libraries the same way as before, assuming that $HOME/.local/lib contains only libc++.so.
I'm trying to do this like: clang++ -L$HOME/.local/lib -lc++, but the compiler still links against /usr/lib64/libc++.so.
How to force the compiler (or linker) to link against a specific library location?
-L adds the directory to the search path used by the linker. This has no effect on the search paths used at runtime. At runtime, the search paths, in order, are:
The environment variable LD_LIBRARY_PATH
rpath specified in the executable
System library path
While you can achieve what you want by specifying the environment variable LD_LIBRARY_PATH=$HOME/.local/lib, it is a bad solution since it modifies the search paths of all executables. Specifying the rpath is a much cleaner solution since it only affects the behaviour of your executable. You can do this through the linker option of your toolchain, which is likely -rpath. So the command would be clang++ -rpath $HOME/.local/lib -lc++.
On either Ubunutu 12.04 or Springdale 6.4, using gcc and g++, what's the difference between C_INCLUDE_PATH (or CPLUS_INCLUDE_PATH) and LD_LIBRARY_PATH? Is the LD one only used at run-time, and the other two only at compile-time?
Since the INCLUDE and LIBRARY_PATH environment variables seem to be ignored by GCC on these operating systems, which should I set when constructing my ~/.bashrc file to make it as portable as possible (modulo changes in the actual paths) across modern Linux OSes?
LD_LIBRARY_PATH is an environment variable which tells in which directories the dll loader should look for dynamic libraries when you start an executable. The variable is dangerous and deprecated
LIBRARY_PATH - tells linker where too look for libraries while building exe or lib
INCLUDE_PATH - tells where to look for files referred in #include statements
In any case, LIBRARY_PATH and INCLUDE_PATH should be set in a particular build-system, not in bashrc. The easier a script can build c-sources, the more probable your PC may be infected with a rootkit.
BTW: gcc is a wrapper which invokes a proper compiler (e.g. cc or g++) and linker.
g++ is gnu c++ compiler
EDIT
Explanation, why LD_LIBRARY_PATH is dangerous.
I haven't used Linux for a couple of years and I wonder, that this env-variable is still in current distributions. It was considered as deprecated when I was using Linux (around 2006) as it provides very easy to exploit hook.
The problem with it is, it prescribes the order of path's in which ld.so - dynamic linker looks for required libraries. If LD_LIBRARY_PATH contained a writable directory, a hacker (in new speech a cybercriminal) could place in that directory a library with a name likely to be found in a system directory (e.g. /usr/lib). This library could first do any dirty job and then call the original library. Exploiting LD_LIBRARY_PATH is much easier then compromising binaries in system directories. And also such an exploit is hard to detect.
I have a c++ shared library which as part of its normal behaviour fork()/execs() another executable containing some unstable legacy code. This executable is not useful other than with this library, so I'd like to avoid placing it in a PATH directory. I'd also like to be able to install multiple copies in various locations, so hard coded paths are not desirable. Is there anything equivalent to a RPATH that will allow exec() to find this executable? Alternatively, is it possible to query the rpath of a shared library from the library itself?
Edit: This post suggests the latter is possible. I'll leave this open in case anybody knows the answer to the asked question. Is there a way to inspect the current rpath on Linux?
You can always use getenv to get the environment within the shared object, but is RPATH really what you want to use for that? Wouldn't it be better to have the shared object have some sort of configuration file in the user's home directory (or custom environment variable) that tells it which location to use run the external binary?
I think the best way to do this is to set an environment variable and use execve() to run the binary. Presumably you could just set PATH and then execve() a shell that would use PATH to find a copy of the executable. The library equivalent would be to set LD_LIBRARY_PATH and execve() a binary that has this library as a dependency.
In either case, you are not changing the external environment, only manufacturing a modified copy that is used with execve().
I have a software properly installed on Kubuntu.
Now, I am patching and testing some of its libraries.
How can I start the software from bash so that it loads my patched libraries instead of the official libs?
e.g.:
the official libs are locate in /usr/lib/
my patch libraries (used during test development) are in /home/user/dev/lib/
I tried:
$ set LD_LIBRARY_PATH=/home/user/dev/lib/
$ binary_app &
but to no avail.
I'd prefer a solution that can be set from the bash, but if it's not possible, I could also modify the cmake file of this C++ software.
The aim is to allow me to easily start the application either with the vanilla libs, or with my patched libs to see the differences.
Edit: it's a KDE .so file
The library I am testing is a KDE4 library. The official lib is in /usr/lib/kde4/ . In that directory, none of the library start with the lib prefix.
Whether I do:
/lib/ld-linux-x86-64.so.2 --list --library-path PATH EXEC
or
ldd EXEC
The library is not listed at all.
On the other hand, if if move the original library away from /usr/lib/kde4/, the application starts but the corresponding functionality is missing.
Are KDE4 libraries loaded in a specific way? Maybe the variable to set is different...
Edit 2
All the answers are good and useful... unfortunately, it turned out that the problem does not appear to be related to the lib path setting. I'm dealing with a plugin architecture and the .so loading path appears to be hard-coded somewhere in the application. I need to spend more time within the source code to understand what's happening... Thanks and +1 to all.
From 'man bash':
When a simple command other than a builtin or shell function is to
be executed, it is invoked in a
separate execution environment that
consists of the following. Unless
otherwise noted, the values are
inherited from the shell.
[....]
ยท shell variables and functions marked for export, along
with variables exported for the
command, passed in the environment
You need to 'export' a variable if it is to be seen by programs you execute.
However, you can also try the following:
/lib/ld-linux.so.2 --library-path PATH EXECUTABLE
See http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
Try export LD_LIBRARY_PATH=... instead of set.
I already put this in a comment but after thinking about it I think the best way to do this (using a different library just for testing/debugging) is using LD_PRELOAD, see What is the LD_PRELOAD trick?
From the man page:
LD_PRELOAD
A whitespace-separated list of additional, user-specified, ELF shared libraries to be loaded before all others. This can be used to selectively override functions in other shared libraries. For set-user-ID/set-group-ID ELF binaries, only libraries in the standard search directories that are also set-user-ID will be loaded.
Update:
After the updated question it seems the application is using dlopen to open the library using a absolute path. I don't think you can do anything about it. See man dlopen
Update2:
Maybe there is something you can do: you might be able to LD_PRELOAD your own dlopen function which modifies the path to your own library...
Isn't you app setuid or setgid by chance? In this case LD_LIBRARY_PATH will be ignored.
Put everything on one line:
LD_LIBRARY_PATH=foo binary_app&