I am working on a system with a system wide glibc 2.27 and am using pathelf to patch some executable to a customized version of glibc 2.23. The executables are compiled with a custom version of LLVM. This has been working fine on C programs, but doesn't work on C++ programs.
# cp Xalan_base.cc-v2 Xalan_base.cc-v2_patched
# /usr/bin/patchelf --set-interpreter /path_to/glibc-2.23_install/lib/ld-2.23.so --set-rpath /path_to/glibc-2.23_install/lib Xalan_base.cc-v2_patched
# ./Xalan_base.cc-v2_patched -v t5.xml xalanc.xsl
/path_to/Xalan_base.cc-v2_patched: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory
# ldd Xalan_base.cc-v2
linux-vdso.so.1 (0x00007ffe9efb8000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8db1524000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8db1186000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8db0f6e000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8db0b7d000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8db18ad000)
# ldd Xalan_base.cc-v2_patched
linux-vdso.so.1 (0x00007ffe08b7c000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f43a5af0000)
libm.so.6 => /path_to/glibc-2.23_install/lib/libm.so.6 (0x00007f43a57eb000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f43a55d3000)
libc.so.6 => /path_to/glibc-2.23_install/lib/libc.so.6 (0x00007f43a5233000)
/path_to/glibc-2.23_install/lib/ld-2.23.so => /lib64/ld-linux-x86-64.so.2 (0x00007f43a5e79000)
# ls -lah /usr/lib/x86_64-linux-gnu/libstdc++.so.6
lrwxrwxrwx 1 root root 19 May 8 08:51 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 -> libstdc++.so.6.0.25
As I understand, libstdc++ is a compiler library. I'm not seeing any paths to my LLVM installation here, and those I do see all appear valid.
Can you explain why this is happening? What do I need to do?
Can you explain why this is happening?
Your /path_to/glibc-2.23_install/lib/ld-2.23.so does not look in /usr/lib/x86_64-linux-gnu, and thus doesn't find /usr/lib/x86_64-linux-gnu/libstdc++.so.6.
You should learn to never use ldd on your patched binary, as it will lie to you. Instead, use /path_to/glibc-2.23_install/bin/ldd, and it should tell you the same thing (that it can't find libstdc++.so.6).
What do I need to do?
You need to arrange for libstdc++.so.6 to be found someplace where /path_to/glibc-2.23_install/lib/ld-2.23.so is actually looking.
Something like:
ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /path_to/glibc-2.23_install/lib
should do the trick. You will likely need to repeat this for libgcc_s.so.1 as well.
Related
I am trying to make sense of how output of ldd --version and ldd -v a.out
I have the below simple program
#include <iostream>
#include <string>
#include <cstring>
int main()
{
std::cout << "Hello world" << std::endl;
std::string a = "Test string";
char b[15] = {};
memcpy(b, a.c_str(), 15);
std::cout << b << std::endl;
return 0;
}
I compile it with following command
g++ --std=c++17 test.cpp
I want to find out which glibc version this program is going to use when I run say memcpy.
The output of ldd --version on this system is:
ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
The output of ldd -v a.out is
ldd -v a.out
linux-vdso.so.1 (0x00007ffe7d3f3000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f050bb2f000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f050bb14000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f050b922000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f050b7d3000)
/lib64/ld-linux-x86-64.so.2 (0x00007f050bd3a000)
Version information:
./a.out:
libgcc_s.so.1 (GCC_3.0) => /lib/x86_64-linux-gnu/libgcc_s.so.1
libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
libstdc++.so.6 (GLIBCXX_3.4.21) => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
libstdc++.so.6 (CXXABI_1.3) => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
libstdc++.so.6 (GLIBCXX_3.4) => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
/usr/lib/x86_64-linux-gnu/libstdc++.so.6:
libm.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libm.so.6
ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
libgcc_s.so.1 (GCC_4.2.0) => /lib/x86_64-linux-gnu/libgcc_s.so.1
libgcc_s.so.1 (GCC_3.4) => /lib/x86_64-linux-gnu/libgcc_s.so.1
libgcc_s.so.1 (GCC_3.3) => /lib/x86_64-linux-gnu/libgcc_s.so.1
libgcc_s.so.1 (GCC_3.0) => /lib/x86_64-linux-gnu/libgcc_s.so.1
libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.6) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.18) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.16) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.17) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libgcc_s.so.1:
libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libc.so.6:
ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/libm.so.6:
ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_PRIVATE) => /lib/x86_64-linux-gnu/libc.so.6
What I can not understand is that if ldd --version says that I have GLIBC version 2.31 available then why did my executables ldd output say GLIBC_2.4 and GLIBC_2.2.5 for a.out.
What is the right way to understand this?
What would happen if I have compiled a binary on a system that has old version of libc.so (suppose has highest version of GLIBC as 2.17) and then run the binary on a system with new version of libc.so (suppose has highest version of GLIBC as 2.31) ?
Thanks
You should read this answer, and look at the output from readelf -V a.out.
When a program is linked, it records the symbol version(s) used (current) at the time of the link.
Many of the symbols your program is using have not changed since e.g. GLIBC_2.2.5, so ldd says: you need at least version GLIBC_2.2.5 (for these symbols). Some of the symbols you use have changed in GLIBC-2.16, GLIBC-2.17 and GLIBC-2.18, so ldd says that you need these as well.
What would happen if I have compiled a binary on a system that has old version of libc.so (suppose has highest version of GLIBC as 2.17) and then run the binary on a system with new version of libc.so (suppose has highest version of GLIBC as 2.31) ?
The recorded symbols (encoded into a.out) will all be GLIBC_2.17 or older, and the program will run fine on a newer system, because GLIBC guarantees backward compatibility (programs built on an older system continue to run fine on newer ones).
But if you did the inverse -- built on a GLIBC-2.31 system and tried to run the program on a GLIBC-2.17 one, it may (or may not, depending on which symbols it actually uses) fail.
In the example you provided, the highest required version of GLIBC is GLIBC_2.18. Therefore, this particular a.out will work fine on a GLIBC-2.18 or newer system, but will fail on GLIBC-2.17 or older one.
Update:
What happens if an executable that is compiled on an old system which has highest version of GLIBC_2_17, is run on a system that has GLIBC_2_31 available? Will the executable pick the latest symbols (if they are ABI complaint) eg say memcpy - exec compiled on old system (with GLIBC without vector support) when run on new system (with GLIBC that has memcpy with vector support) will pick memcpy from new GLIBC that has vector support.
Yes, the executable will pick up the version it was linked with, and so long as the ABI for a given function hasn't changed, it will be the most recent version.
The case of memcpy in particular is a bit more complicated. On Fedora 35 x86_64 system (GLIBC-2.34):
nm /lib64/libc.so.6 | grep ' memcpy'
00000000000a1560 i memcpy
00000000000a1560 i memcpy##GLIBC_2.14
00000000000b9c30 T memcpy#GLIBC_2.2.5
What you can see here is that the memcpy ABI changed in GLIBC-2.14, and it became a GNU indirect function. You can read the details at the link, but TL;DR is that the actual function called by your program will depend on processor capabilities. It could be any one of these:
00000000001870b0 t __memcpy_avx512_no_vzeroupper
0000000000189f00 t __memcpy_avx512_unaligned
0000000000189f70 t __memcpy_avx512_unaligned_erms
00000000001828f0 t __memcpy_avx_unaligned
0000000000182960 t __memcpy_avx_unaligned_erms
000000000018b0e0 t __memcpy_avx_unaligned_erms_rtm
000000000018b070 t __memcpy_avx_unaligned_rtm
00000000000b9ca0 t __memcpy_erms
00000000001920b0 t __memcpy_evex_unaligned
0000000000192120 t __memcpy_evex_unaligned_erms
00000000000b9c30 t __memcpy_sse2_unaligned
00000000000b9d10 t __memcpy_sse2_unaligned_erms
000000000015d400 t __memcpy_ssse3
0000000000162990 t __memcpy_ssse3_back
I just built GCC 5.1 on Ubuntu 14.04, which has gcc 4.8 as default. When I try to build things with it, I find that ld will use the default libstdc++ instead of the newly build one.
Here is the output:
drizzlex#dx ~/test
$ g++ hello.cpp
drizzlex#dx ~/test
$ ldd a.out
linux-vdso.so.1 => (0x00007ffde0d25000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa181ad2000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa1817cc000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa1815b5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa1811f0000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa181dfd000)
And if I use $ export LD_LIBRARY_PATH=/usr/local/lib64/, it will find the right one.
drizzlex#dx ~/test
$ ldd a.out
linux-vdso.so.1 => (0x00007fffeeaf5000)
libstdc++.so.6 => /usr/local/lib64/libstdc++.so.6 (0x00007f4583d92000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4583a67000)
libgcc_s.so.1 => /usr/local/lib64/libgcc_s.so.1 (0x00007f4583850000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f458348b000)
/lib64/ld-linux-x86-64.so.2 (0x00007f458410e000)
I would like to know what should I do to make it correct? Since I know set LD_LIBRARY_PATH is not the best choice.
For building with g++ 5.1 use this:
$ g++5.1 hello.cpp -Wl,-rpath,/usr/local/lib64
And you will not need set LD_LIBRARY_PATH.
This is from https://en.wikipedia.org/wiki/Rpath
rpath is a term in programming which refers to a run-time search path
hard-coded in an executable file or library, used during dynamic
linking to find the libraries the executable or library requires.
I know there is a lot about this on the internet and I have tried most of it without luck. Most solutions says that en environment variable is missing aka (LD_LIBRARY_PATH) which i pointed it to the file in user and root but it still does not pick it up. I am not sure what exactly is looking for the library or how to fix this??
Any ideas?
EDIT
ldd output:
linux-vdso.so.1 => (0x00007fffb97ff000)
libmysqlcppconn.so.7 => not found
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f002fdb3000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f002fb9c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f002f7dd000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f002f5c0000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f002f2c3000)
/lib64/ld-linux-x86-64.so.2 (0x00007f00300c7000)
Here's how you correctly set the library path...
If your library is at /usr/local/lib/libmylib.so.4 and your program is ./myprog:
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib ./myprog
I'm trying to Hoard allocator to work, but it seems it doesn't. I have a benchmark application that does a lot of dynamic memory management. The execution time for Hoard and glibc memory manager is the same. It makes me wonder if I'm doing the right thing.
What I do is...
export LD_PRELOAD="/path/libhoard.so"
g++ main.cpp -O3 -o bm -lpthread -lrt
Shouldn't I have to link to Hoard allocator? Does it matter what path (in LD_PRELOAD) is, or can I have whatever path?
I'm running Ubuntu 8.04, and g++ 4.2.4
Cheers
No one knows any Linux command (such as grep) to find out if Hoard is loaded properly, and is the actual allocator used?
Author of Hoard here.
(a) Any path for LD_PRELOAD is fine (as long as it is correct).
(b) To see whether your code is using Hoard or not, use the ldd command. If everything went according to plan, then you will see the Hoard library (notice the second line after the second invocation of ldd).
Best,
-- Emery Berger
bash-3.2$ ldd /bin/ls
linux-vdso.so.1 => (0x00007fffe6dfd000)
librt.so.1 => /lib64/librt.so.1 (0x0000003159600000)
libacl.so.1 => /lib64/libacl.so.1 (0x000000315e200000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x000000315d200000)
libc.so.6 => /lib64/libc.so.6 (0x0000003154e00000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003155a00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003154a00000)
libattr.so.1 => /lib64/libattr.so.1 (0x0000003162000000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003155600000)
libsepol.so.1 => /lib64/libsepol.so.1 (0x000000315ce00000)
bash-3.2$ export LD_PRELOAD=$PWD/libhoard.so
bash-3.2$ ldd /bin/ls
linux-vdso.so.1 => (0x00007fff24bfd000)
/nfs/cm/users1/emery/scratch/projects/hoard/trunk/src/libhoard.so (0x00002b4885f42000)
librt.so.1 => /lib64/librt.so.1 (0x0000003159600000)
libacl.so.1 => /lib64/libacl.so.1 (0x000000315e200000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x000000315d200000)
libc.so.6 => /lib64/libc.so.6 (0x0000003154e00000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003155600000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003155a00000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000315b200000)
libm.so.6 => /lib64/libm.so.6 (0x0000003155200000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x000000315aa00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003154a00000)
libattr.so.1 => /lib64/libattr.so.1 (0x0000003162000000)
libsepol.so.1 => /lib64/libsepol.so.1 (0x000000315ce00000)
bash-3.2$
I'm working though the First Steps tutorial on the POCO Project site, and I've successfully built the library (Debian Linux, 2.6.26, gcc 4.3.2) under my home directory
~/Development/POCO
with the shared libraries located in
~/Development/POCO/lib/Linux/x86_64/lib
My problem is that any application I build that depends on these libraries can only be run from the shared library directory.
~/Development/POCO/lib/Linux/x86_64$ ldd ~/Development/Cloud/DateTimeSample/bin/Linux/x86_64/DateTime
linux-vdso.so.1 => (0x00007fffe69fe000)
libPocoFoundation.so.6 (0x00007fa8de44f000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fa8de233000)
libdl.so.2 => /lib/libdl.so.2 (0x00007fa8de02f000)
librt.so.1 => /lib/librt.so.1 (0x00007fa8dde26000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fa8ddb1a000)
libm.so.6 => /lib/libm.so.6 (0x00007fa8dd897000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007fa8dd680000)
libc.so.6 => /lib/libc.so.6 (0x00007fa8dd32d000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa8de7e0000)
And running DateTime from this directory would work as you would expect. However
~/Development/Cloud/DateTimeSample/bin/Linux/x86_64$ ldd DateTime
linux-vdso.so.1 => (0x00007fff24dfe000)
libPocoFoundation.so.6 => not found
libpthread.so.0 => /lib/libpthread.so.0 (0x00007ffc1c7dd000)
libdl.so.2 => /lib/libdl.so.2 (0x00007ffc1c5d9000)
librt.so.1 => /lib/librt.so.1 (0x00007ffc1c3d0000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007ffc1c0c4000)
libm.so.6 => /lib/libm.so.6 (0x00007ffc1be41000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007ffc1bc2a000)
libc.so.6 => /lib/libc.so.6 (0x00007ffc1b8d7000)
/lib64/ld-linux-x86-64.so.2 (0x00007ffc1c9f9000)
so running the executable from any other directory results in
error while loading shared libraries: libPocoFoundation.so.6: cannot open shared object file: No such file or directory
Looking at the output from the make process, the directory is correctly specified
g++ [blah] -L/home/npalko/Development/POCO/lib/Linux/x86_64
-lPocoFoundation
I've tried setting LD_LIBRARY_PATH to /home/npalko/Development/POCO/lib/Linux/x86_64, but it has not changed anything. Any help would be greatly appreciated!
If you don't want to have to deal with the LD_LIBRARY_PATH variable, you can add the linker -rpath option to the gcc command line. In your case, this would be:
gcc ... -Wl,-rpath=/home/npalko/Development/POCO/lib/Linux/x86_64
This effectively hardcodes that path in the executable, so it may or may not be suitable for your purposes.
This fails?
LD_LIBRARY_PATH=/home/npalko/Development/POCO/lib/Linux/x86_64 ~/Development/Cloud/DateTimeSample/bin/Linux/x86_64/DateTime
Just thought you may not be setting LD_LIBRARY_PATH properly
And this?
ls -alh /home/npalko/Development/POCO/lib/Linux/x86_64/libPocoFoundation.so
If both fail I can't see a reason.
You must specify to the linker the path of your library
g++ [blah] -Wl,-rpath=/home/npalko/Development/POCO/lib/Linux/x86_64
-Wl means your pass an option to the linker
-rpath is the linker option
Add a directory to the runtime library search path. This is used
when linking an ELF executable with shared objects. All -rpath
arguments are concatenated and passed to the runtime linker, which
uses them to locate shared objects at runtime. The -rpath option
is also used when locating shared objects which are needed by
shared objects explicitly included in the link;