Why dynamic linked binary shows hard coded SO name? - c++

Sorry it might be a stupid question.
I didn't link libaudit directly, but why its name shows in my binary?
Strings shows:
strings dataserver|grep libaudit
libaudit.so.0
readelf shows:
readelf -a dataserver|grep "Shared lib"
0x0000000000000001 (NEEDED) Shared library: [libsapcrypto.so]
0x0000000000000001 (NEEDED) Shared library: [libaio.so.1]
0x0000000000000001 (NEEDED) Shared library: [libnsl.so.1]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libpam.so.0]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [libaudit.so.0]
What does it mean and is there anyway I can check why it is added into my binary?
Thanks
Update
Thanks. I'm reading the links below and adding some more findings.
I find that libpam was linked and it requires libaudit.so.0
Check libpam.so.0
ldd /lib64/libpam.so.0
linux-vdso.so.1 => (0x00007ffff7fdf000)
libaudit.so.0 => /lib64/libaudit.so.0 (0x00007ffff7b80000) <-- libaudit
libdl.so.2 => /lib64/libdl.so.2 (0x00007ffff797c000)
libc.so.6 => /lib64/libc.so.6 (0x00007ffff7616000)
/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fe0000)
However, I have 2 binaries, both linked libpam.so. For binary1, it list libaudit.so.0 as NEEDED, but for binary2, it doesn't.
binary1 (with libpam but not depends on libaudit.so.0):
readelf -d binary1|grep "Shared lib"
0x0000000000000001 (NEEDED) Shared library: [libsybcsi_core210.so]
0x0000000000000001 (NEEDED) Shared library: [libsybcsi_profiler210.so]
0x0000000000000001 (NEEDED) Shared library: [libsybcsi_propertiesconfig210.so]
0x0000000000000001 (NEEDED) Shared library: [libsybcsi_openssl210.so]
0x0000000000000001 (NEEDED) Shared library: [libaio.so.1]
0x0000000000000001 (NEEDED) Shared library: [libnsl.so.1]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libpam.so.0] <--- libpam
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
binary2 (with libpam but and depends on libaudit.so.0):
readelf -d binary2|grep "Shared lib"
0x0000000000000001 (NEEDED) Shared library: [libsapcrypto.so]
0x0000000000000001 (NEEDED) Shared library: [libaio.so.1]
0x0000000000000001 (NEEDED) Shared library: [libnsl.so.1]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libpam.so.0] <--- libpam
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [libaudit.so.0] <--- libaudit
It confused me a lot about the key word "NEEDED", when will a lib is "NEEDED" if it is not directly used?

What does it mean
It means your binary depends on it.
and is there anyway I can check why it is added into my binary?
Most likely the linker you used found that some other library on the link line depends on libaudit, and automagically added it in.
GNU-ld does this, and Gold doesn't (producing a steady stream of programs that have incorrect Makefiles and fail to link with Gold).
If you link with -v flag, you should see the actual link command, although it may be hidden by collect2 or some such wrapper.
Running the link under strace -fvs 1024 -e execve will show the actual link command for sure.

All right, I find the cause.
It's due to some reference of "_edata" and "_end" in my code.
In libpam.so, it is not found but libaudit.so which is needed for libpam.so, it was found and linker resolved it, then add libaudit as "NEEDED".
And _end is exposed as follows from linker scripts –
_end = .; PROVIDE (end = .);
This means, we should be using “end” and not “_end” unless we PROVIDE “_end” with our own linker scripts.
So, the fix is to change _end (and all such related symbols like _etext and _edata) to end (and etext, edata) so that they will be resolved correctly from standard libc, avoiding any dependency on other shared objects like libaudit.so.

Related

C++ executable fail to look up LD_LIBRARY_PATH

I am building a simple C++ benchmark using a custom compilation tool (which has in-built clang plugin) and it simply is another regular C++ compiler. However whenever I try to run the compiled (and linked) binary, it never looks up for LD_LIBRARY_PATH to find the correct libstdc++ version. It always look for the standard /usr/lib64 and the libstdc++ version there is too old for my executable. I want to point my binary to look up in LD_LIBRARY_PATH when it runs. (I don't have root access.)
Below is the output when running it:
[xx#login-01 src]$ ./executable.x
./executable.x: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./executable.x)
Below is output from gcc --version command. This GCC version is sufficient but my executable doesn't look for its lib path, which is in LD_LIBRARY_PATH.
gcc (GCC) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
echo $LD_LIBRARY_PATH output is below. Notice it has correct GCC lib path.
/cm/local/apps/gcc/9.2.0/lib:/cm/local/apps/gcc/9.2.0/lib64:/home/ri-wshilpage/tools/LLVM/llvm-project/build/lib:/home/ri-wshilpage/tools/LLVM/llvm-project/build/lib/libomp.so:/opt/gcc/8.1.0/snos/lib64:/opt/cray/pe/papi/6.0.0.7/lib64:/cm/shared/apps/pbspro/19.2.8.20200925072630/lib/
Below I have checked the output from readelf -d ./executable.x and its RPATH shows only /usr/lib64 and the lib path of the custom compiler.
Dynamic section at offset 0x92d58 contains 34 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libhipSYCL-rt.so]
0x0000000000000001 (NEEDED) Shared library: [libboost_context.so.1.66.0]
0x0000000000000001 (NEEDED) Shared library: [libboost_fiber.so.1.66.0]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libomp.so]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000f (RPATH) Library rpath: [/lustre/home/xx/sycl-compilers/hipSYCL/build/install/bin/../lib/:/usr/lib64]
0x000000000000000c (INIT) 0x407000
0x000000000000000d (FINI) 0x477d98
0x0000000000000019 (INIT_ARRAY) 0x493980
0x000000000000001b (INIT_ARRAYSZ) 160 (bytes)
0x000000000000001a (FINI_ARRAY) 0x493a20
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x0000000000000004 (HASH) 0x400320
0x000000006ffffef5 (GNU_HASH) 0x400b70
0x0000000000000005 (STRTAB) 0x402798
0x0000000000000006 (SYMTAB) 0x400e90
0x000000000000000a (STRSZ) 12941 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000015 (DEBUG) 0x0
0x0000000000000003 (PLTGOT) 0x494000
0x0000000000000002 (PLTRELSZ) 4056 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x405fe8
0x0000000000000007 (RELA) 0x405df0
0x0000000000000008 (RELASZ) 504 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x405c40
0x000000006fffffff (VERNEEDNUM) 6
0x000000006ffffff0 (VERSYM) 0x405a26
0x0000000000000000 (NULL) 0x0
And finally here's the Makefile.
#-----------------------------------------------------------------------
# This file compiles for OpenMP and MPI hybrid operations using the GNU
# compile chain.
MINIFE_TYPES = \
-DMINIFE_SCALAR=double \
-DMINIFE_LOCAL_ORDINAL=int \
-DMINIFE_GLOBAL_ORDINAL=int \
-DMINIFE_RESTRICT=__restrict__
MINIFE_MATRIX_TYPE = -DMINIFE_CSR_MATRIX
#-----------------------------------------------------------------------
CXX = syclcc
CC = syclcc
OPTIMIZE = yes
CPPFLAGS = --gcc-toolchain=/lustre/projects/bristol/modules/gcc/9.2.0 --hipsycl-targets=omp -I. -I../utils -I../fem $(MINIFE_TYPES) -I../../include/ $(MINIFE_MATRIX_TYPE) \
-std=c++17 -I/home/xx/tools/LLVM/llvm-project/build/projects/openmp/runtime/src
ifeq ($(OPTIMIZE),yes)
CFLAGS += -O3
endif
CXXFLAGS = $(CFLAGS)
include make_targets
Also note that I have built LLVM from source (version 12.0.1) and used it to build my custom compiler. Everything works fine except for the executable not looking up at LD_LIBRARY_PATH for shared libraries.
Edit:
Check the output from ldd ./executable.x below.
./executable.x: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./executable.x)
linux-vdso.so.1 (0x0000155555551000)
libhipSYCL-rt.so => /lustre/home/xx/sycl-compilers/hipSYCL/build/install/bin/../lib/libhipSYCL-rt.so (0x00001555554ff000)
libboost_context.so.1.66.0 => /usr/lib64/libboost_context.so.1.66.0 (0x0000155555128000)
libboost_fiber.so.1.66.0 => /usr/lib64/libboost_fiber.so.1.66.0 (0x0000155554ec5000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000155554b30000)
libm.so.6 => /usr/lib64/libm.so.6 (0x00001555547ae000)
libomp.so => /home/xx/tools/LLVM/llvm-project/build/lib/libomp.so (0x000015555540d000)
libgcc_s.so.1 => /usr/lib64/libgcc_s.so.1 (0x0000155554596000)
libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x0000155554376000)
libc.so.6 => /usr/lib64/libc.so.6 (0x0000155553fb4000)
libdl.so.2 => /usr/lib64/libdl.so.2 (0x0000155553db0000)
/lib64/ld-linux-x86-64.so.2 (0x000015555532b000)
librt.so.1 => /usr/lib64/librt.so.1 (0x0000155553ba7000)
libboost_filesystem.so.1.66.0 => /usr/lib64/libboost_filesystem.so.1.66.0 (0x000015555398c000)
libboost_system.so.1.66.0 => /usr/lib64/libboost_system.so.1.66.0 (0x0000155553787000)

Integrating shared object in an application, ldd and readelf show different outputs

I am trying to integrate OpenCV into an application and facing some issues below.
error while loading shared libraries: libopencv_imgproc.so.4.1: cannot open shared object file: No such file or directory
However, when I check the output of ldd and readelf, below are the differences.
$readelf -d <app name>
0x0000000000000001 (NEEDED) Shared library: [libopencv_imgcodecs.so.4.1]
0x0000000000000001 (NEEDED) Shared library: [libopencv_core.so.4.1]
$ldd <app name>
libopencv_imgcodecs.so.4.1 => <path to opencv>/opencv/lib/libopencv_imgcodecs.so.4.1 (0x00007f04555b2000)
libopencv_imgproc.so.4.1 => <path to opencv>/opencv/lib/libopencv_imgproc.so.4.1 (0x00007f0453e8f000)
libopencv_core.so.4.1 => <path to opencv>/opencv/lib/libopencv_core.so.4.1 (0x00007f0452774000)
As shown above, libopencv_imgproc is missing from readelf output. Can someone point out what causes such behavior?

Integrate OpenCV using cmake - cannot open shared object file

I am trying to integrate OpenCV (v4.1.2) into an existing project and I made below changes for that:
CMakeLists.txt
find_package(OpenCV REQUIRED core imgcodecs imgproc PATHS <path to all opencv shared object files>)
target_link_libraries(<exec name> ${OpenCV_LIBS})
Also added below to the config file:
#cmakedefine HAVE_LIBOPENCV 1
#cmakedefine HAVE_OPENCV_H 1
Copy the output executable at:
/usr/bin
Run the below command to set rpath:
patchelf --set-rpath path-to-opencv/opencv/lib:$ORIGIN /usr/bin/executablename
Below are the changes I added in the source code:
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
cv::Mat img = cv::imread("sample.jpg", ZM_MAX_IMAGE_COLOURS);
cv::imwrite("sample-opencv.jpg", img);
However, when I run the the code, I see below message in logs:
error while loading shared libraries: libopencv_imgproc.so.4.1: cannot open shared object file: No such file or directory
Below are some sanity checks I performed on the executable file:
$readelf -d <executable name>
0x0000000000000001 (NEEDED) Shared library: [libopencv_imgcodecs.so.4.1]
0x0000000000000001 (NEEDED) Shared library: [libopencv_core.so.4.1]
0x000000000000001d (RUNPATH) Library runpath: [<path to opencv>/opencv/lib:]
$ ldd <executable name>
libopencv_imgcodecs.so.4.1 => <path to opencv>/opencv/lib/libopencv_imgcodecs.so.4.1 (0x00007f49cf6f9000)
libopencv_core.so.4.1 => <path to opencv>/opencv/lib/libopencv_core.so.4.1 (0x00007f49ce4fc000)
libopencv_imgproc.so.4.1 => <path to opencv>/opencv/lib/libopencv_imgproc.so.4.1 (0x00007f49c3b12000)
Even though there is a valid link in ldd output, libopencv_imgproc.so doesn't exist in readelf output. My assumption is that since there is no source code directly consuming imgproc, it doesn't show up in readelf output.
Can someone point out what is causing the "cannot open shared object file" error?
EDIT:
I changed my above code to include imgproc header and declare a class below as shown below:
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/imgproc.hpp>
cv::Subdiv2D div; // comes from imgproc
cv::Mat img = cv::imread("sample.jpg", ZM_MAX_IMAGE_COLOURS);
cv::imwrite("sample-opencv.jpg", img);
It worked after this change. Also, below is the new readelf output:
$ readelf -d <executable name>
0x0000000000000001 (NEEDED) Shared library: [libopencv_imgcodecs.so.4.1]
0x0000000000000001 (NEEDED) Shared library: [libopencv_imgproc.so.4.1]
0x0000000000000001 (NEEDED) Shared library: [libopencv_core.so.4.1]
0x000000000000001d (RUNPATH) Library runpath: [<path to opencv>/opencv/lib:]
However, it is just a workaround. One shouldn't need to write a placeholder code like this. Can someone help figure out a clean solution to include shared objects correctly?
FYI - Here are the internal dependecies between these opencv shared objects:
libopencv_core --> no dependency
libopencv_imgproc --> libopencv_core
libopencv_imcodecs --> libopencv_core and libopencv_imgproc

clang, linux, is there an option to change shared library name at linking?

I'm testing a somewhat non-conventional project layout and rake as make utility. There is a rule to compile binaries from source files in different directories and link them with a shared library. This rule is run from the root directory of the project. For instance the rule does this:
clang -I libs/ -o tests/sourcefile2 tests/sourcefile2.c shared_libs/libFoo.so
And as a result I get the full path shared_libs/libFoo.so in the binary:
readelf -d tests/sourcefile2
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [shared_libs/libFoo.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
...
I would like to change it to just 'libFoo.so' like this:
readelf -d tests/sourcefile2
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libFoo.so]
...
Then I could set RPATH for dynamic linker as I want and it would give some flexibility. But I cannot find the corresponding option or similar example. Could you suggest how to handle this? Should I just use a temporary directory for the build, copy everything and compile there?
Not sure if it will help you. But when I try compile some shit and I don't know what flags. I use pkg-config.
For example, to compile a program which uses Xlib
pkg-config -cflags -libs x11
and the output is the following
-I/usr/X11R7/include -D_REENTRANT -Wl,-rpath,/usr/X11R7/lib -L/usr/X11R7/lib -lX11
Note this vary on systems, for example NetBSD forces me to link it with rpath, and there are optional arguments in this output.
So I copy the output of pkg-config and it compiles.
if If you use 'ld' as your linker you should be able to use "-Wl,-soname ".

FMOD Debian libfmod.so.8: cannot open shared object file: No such file or directory

I'm trying to install and validate fmod on my raspberry pi 1 model b with debian by running the provided example program play_stream. The compilation works fine however when I try to run the built executable it fails with the error
error while loading shared libraries: libfmod.so.8: cannot open shared object file: No such file or directory
I'm unsure of what i'm doing wrong. Any help or advice you can provide would be great
My set up:
~/fmodstudioapi10813linux/api/lowlevel/examples/make $ make --file play_stream.makefile CONFIG=Debug CPU=arm
~/fmodstudioapi10813linux/api/lowlevel/examples/make $ ./play_stream
./play_stream: error while loading shared libraries: libfmodL.so.8: cannot open shared object file: No such file or directory
~/fmodstudioapi10813linux/api/lowlevel/lib/arm $ ls /usr/local/lib/
libfmodL.so libfmodL.so.8 libfmodL.so.8.13 libfmod.so libfmod.so.8 libfmod.so.8.13
~ $ ls /usr/local/include/
fmod_codec.h fmod_common.h fmod_dsp_effects.h fmod_dsp.h fmod_errors.h fmod.h fmod.hpp fmod_output.h node
~ $ env | grep '^LD_LIBRARY_PATH'
LD_LIBRARY_PATH=:/usr/local/lib:/home/pi/fmodstudioapi10813linux/api/lowlevel/lib/arm
~/fmodstudioapi10813linux/api/lowlevel/examples/make $ ldd play_stream
/usr/lib/arm-linux-gnueabihf/libarmmem.so (0xb6faf000)
libfmod.so.8 => not found
libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb6ec0000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6e45000)
libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6e18000)
libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xb6df0000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6caf000)
/lib/ld-linux-armhf.so.3 (0x7f56d000)
~/fmodstudioapi10813linux/api/lowlevel/examples/make $ echo $LD_LIBRARY_PATH
:/usr/local/lib:/home/pi/fmodstudioapi10813linux/api/lowlevel/lib/arm
**~/fmodstudioapi10813linux/api/lowlevel/examples/make $** readelf -d play_stream
Dynamic section at offset 0x420c contains 30 entries:
Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libfmod.so.8]
0x00000001 (NEEDED) Shared library: [libstdc++.so.6]
0x00000001 (NEEDED) Shared library: [libm.so.6]
0x00000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x00000001 (NEEDED) Shared library: [libpthread.so.0]
0x00000001 (NEEDED) Shared library: [libc.so.6]
0x0000000f (RPATH) Library rpath: [$ORIGIN/../../../lowlevel/lib/arm/]
Run ldd play_stream and echo $LD_LIBRARY_PATH.