As we all know,gcc will search some default dir when compiling targets,
I use gcc -print-search-dirs commnd and get this stuff:
install: /usr/lib/gcc/x86_64-redhat-linux/4.8.5/
programs: =/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/bin/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/bin/
libraries: =/usr/lib/gcc/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/lib/x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../x86_64-redhat-linux/4.8.5/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/:/lib/x86_64-redhat-linux/4.8.5/:/lib/../lib64/:/usr/lib/x86_64-redhat-linux/4.8.5/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/lib/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../:/lib/:/usr/lib/
I have two questions about this:
What does /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../x86_64-redhat-linux/bin/ mean, and why write like that?
Why /usr/include is not there?Doesn't gcc need to know where headfile is?
So as to question 1, I found this in the gcc mailing list:
Note that a/b/c/../../../x will only exist if a/b/c exists.
So your gcc will only search in /usr/x86_64-redhat-linux/bin/ if /usr/lib/gcc/x86_64-redhat-linux/4.8.5/ exists.
Note that I've got similar search-dirs on arch, and the /usr/x86_64-redhat-linux/-equivalent ones don't exist, so I'm not sure WHY they are even included, but Linux has a notoriously unstandardized directory structure, so perhaps it's important for other distros.
Regarding 2, from the man page:
-print-search-dirs
Print the name of the configured installation directory and a list of program and library directories gcc searches---and don't do anything else.
This is useful when gcc prints the error message installation problem, cannot exec cpp0: No such file or directory. To resolve this you either need to put cpp0 and the other
compiler components where gcc expects to find them, or you can set the environment variable GCC_EXEC_PREFIX to the directory where you installed them. Don't forget the
trailing /.
This seems to print not the library paths, but rather the paths where gcc expects to find the components it calls to do it's work.
To find the paths the preprocessor searches, type
`gcc -print-prog-name=cc1plus` -v
or
`gcc -print-prog-name=cc1` -v
for the C preprocessor.
Related
As part of a research project I'm trying to use clang 6.0.1 with Xcode 9.4.1. I've built and installed clang in a custom location (/opt/llvm-6_0_1/clang). I wrote a simple xcplugin compiler specification to integrate my clang version with Xcode.
Now I can open projects in Xcode, select my proxy compiler and use it to build instead of Apple's default clang.
There were some minor additions that I had to make to the xcplugin's xcspec file to get this to work that probably won't be interesting to most people, so I won't provide the details here unless asked.
This all works with most of the projects I've played with, but I'm running into an odd problem where an implicitly linked static library cannot be found by my copy of clang. Specifically I get this error:
ld: file not found: /opt/llvm-6_0_1/clang/Toolchains/LLVM6.0.1.xctoolchain/usr/lib/arc/libarclite_macosx.a
clang-6.0: error: linker command failed with exit code 1 (use -v to see invocation)
Note that the libarclite_macosx.a file is not explicitly included by the Xcode project. I figured it must be implicitly included, perhaps because this project enables ARC?
After pouring over the Xcode generated link command line (it's complex) I decided to look at the MyProject__dependency_info.dat file, which is passed in via the -dependency_info option. Apparently this data file (the path is defined as env var LD_DEPENDENCY_INFO_FILE) is created during the linking process, not as an input to the linker. Perhaps it exists because of a hack workaround using symlinks that I used to get a link to work (described at the end).
In any case the format appears to be binary, but I was able to see a text reference to libarclite_macosx.a in the file:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_macosx.a
After enabling the -Xlinker -v option I could see that my built clang was not searching the default toolchain lib or arc paths so I added them:
-L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
-L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc
Now I can see the search paths in the verbose output, but clang still cannot find the library:
Library search paths:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc
I've tried adding the paths to the frameworks search paths. I also tried defining the various link path env vars. Nothing has worked.
To try to get a sense of what clang is actually doing, I used fs_usage while getting the link error:
sudo fs_usage -e -w -f filesys | grep "lib/arc"
14:11:00.461965 stat64 [ 2] /opt/llvm-6_0_1/clang/Toolchains/LLVM6.0.1.xctoolchain/usr/lib/arc>>>>>>>>>>>>>>>>>>>>>> 0.000006 ld.1421614
14:11:00.461968 stat64 [ 2] /opt/llvm-6_0_1/clang/Toolchains/LLVM6.0.1.xctoolchain/usr/lib/arc>>>>>>>>>>>>>>>>>>>>>> 0.000002 ld.1421614
Clearly clang really wants to look for this file in the installed location, not the location indicated in the -dependency_info, nor in the search paths that I'm providing.
At this stage the only way I can get a build to work is to add a symlink to Xcode's "arc" directory to my installed clang lib directory. That "works", but is fragile and nasty.
Any thoughts as to how how I can get clang find the static library where it actually lives?
I wrote a custom (cc_)toolchain for Clang. After some back and forth I now have it mostly working. It is however failing with:
undeclared inclusion(s) in rule '//foo:foo'
this rule is missing dependency declarations for the following files included by 'foo/foobar.cpp':
'/usr/include/features.h'
'/usr/include/stdc-predef.h'
'/usr/include/x86_64-linux-gnu/sys/cdefs.h'
'/usr/include/x86_64-linux-gnu/bits/wordsize.h'
'/usr/include/x86_64-linux-gnu/gnu/stubs.h'
'/usr/include/x86_64-linux-gnu/gnu/stubs-64.h'
'/usr/include/stdint.h'
'/usr/include/x86_64-linux-gnu/bits/wchar.h'
'/usr/include/stdio.h'
'/usr/include/x86_64-linux-gnu/bits/types.h'
'/usr/include/x86_64-linux-gnu/bits/typesizes.h'
'/usr/include/libio.h'
'/usr/include/_G_config.h'
'/usr/include/wchar.h'
'/usr/include/x86_64-linux-gnu/bits/stdio_lim.h'
'/usr/include/x86_64-linux-gnu/bits/sys_errlist.h'
'/usr/include/math.h'
'/usr/include/x86_64-linux-gnu/bits/math-vector.h'
'/usr/include/x86_64-linux-gnu/bits/libm-simd-decl-stubs.h'
'/usr/include/x86_64-linux-gnu/bits/huge_val.h'
'/usr/include/x86_64-linux-gnu/bits/huge_valf.h'
'/usr/include/x86_64-linux-gnu/bits/huge_vall.h'
'/usr/include/x86_64-linux-gnu/bits/inf.h'
'/usr/include/x86_64-linux-gnu/bits/nan.h'
'/usr/include/x86_64-linux-gnu/bits/mathdef.h'
'/usr/include/x86_64-linux-gnu/bits/mathcalls.h'
'/usr/include/assert.h'
'/usr/include/string.h'
'/usr/include/xlocale.h'
So how is one supposed to handle these files? Simply hardcode these for the toolchain? Or is there a switch to ignore dependencies from the system (/usr)? Or rather use a hermetic libc like musl, newlib or bionic.
EDIT:
For more details about the actual toolchain see: https://github.com/abergmeier/bazel_toolchains
You need to put the builtin include paths of the compiler into the cxx_builtin_include_directory field of the crosstool proto, so Bazel knows to ignore them during include dependency validation. You can see the builtin include paths of a gcc-compatible compiler by running something like
$ $CC -x c++ -v - -c -o /dev/null < /dev/null
and looking for the output like
#include "..." search starts here:
#include <...> search starts here:
/usr/include/c++/7
/usr/include/x86_64-linux-gnu/c++/7
/usr/include/c++/7/backward
/usr/lib/gcc/x86_64-linux-gnu/7/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
I have a few related questions about my issues with compilation for embedded system. My questions are not only about HOW to do something, but more about WHY, because I have solutions for my problems (but maybe there are better ones?), but have no idea why some things works in some conditions, and does not work in others. I already spent some time with this, but until yesterday I was doing things a little blindly, with trials and errors, and without knowing what I was doing. Time to stop that! Please, help.
Scenario
I want to develop an application for Xilinx’s Zynq ARM processor, on Zedboard. The app will involve multithreading, some audio manipulation, and httpserver. So I will need pthread, alsa, sndfile and microhttpd libraries. I created rootfs with yocto. In original conf.local file I added/modified these lines:
BB_NUMBER_THREADS ?= "${#oe.utils.cpu_count()}"
PARALLEL_MAKE ?= "-j ${#oe.utils.cpu_count()}"
MACHINE ?= "zedboard-zynq7"
PACKAGE_CLASSES ?= "package_deb"
EXTRA_IMAGE_FEATURES = "debug-tweaks eclipse-debug"
IMAGE_INSTALL_append = "libgcc alsa-utils mpg123 libstdc++ sthttpd libmicrohttpd libsndfile1"
LICENSE_FLAGS_WHITELIST = "commercial_mpg123"
I also had to add some additional layers to bblayers.conf (and of course downloaded them):
meta-xilinx
meta-multimedia (from meta-openembedded)
meta-oe (from meta-openembedded)
meta-webserver (from meta-openembedded)
Lastly, I generated core-image-minimal with bitbake.
This, together with Linux kernel, and other stuff compiled separately, boots and works fine.
Problems
1. Simple app with this rootfs
It is app for Zynq, so I use XSDK, which is SDK from Xilinx, based on Eclipse. I created new Application project. In dialog window I chose Linux as platform, C++ as language, and I provided path to my unpacked rootfs (excactly the one that system boots with, via NFS). My rootfs path is /home/stas/ZedboardPetalinuxFS (it is not Petalinux, I just used to use it, and this folder name is still the same). This sets proper paths for library and headers search in rootfs.
I started with something very simple:
#include <pthread.h>
int main()
{
int i;
i = 1;
return 0;
}
I also added pthread library for linker (in Eclipse settings). Linking command at this point:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lpthread
At this point it compiles. But it stops, when I add sndfile library
#include <sndfile.h>
This is reasonable, because this rootfs does not have all headers. I need to add another path for searching for headers. So I added path in yocto tmp folder, that has all the headers, that was needed for building rootfs. After I add it, it compiles again successfully. But problems started, when I added sndfile library for linking. Here is linking command and error:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lpthread -lsndfile
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lsndfile
I looked to usr/lib to check if libsndfile.so is there, and I found only libsndfile.so.1 and ibsndfile.so.1.27. But it is also the case for pthread, and linker does not complain for that. I decided to create libsndfile.so by hand (I linked it to libsndfile.so.1). Linker stopped complaining about it, but started complaining about it’s dependencies. So I also creaded .so files for all the dependencies, and their dependencies, and added them for linking. Then it succeeded. At the end, linking command looked like this:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lpthread -lvorbisenc -lvorbis -logg -lFLAC -lsndfile
So here goes the first question – why I did not needed .so file for pthread, but needed it for all other libraries? Or more general – when do I need .so file, and when .so.X file is enough?
2. Simple app - another approach
After the first try, I thought I should make another image, this time more suitable for development. Luckily, in Yocto it is quite easy – I just had to modify one line:
EXTRA_IMAGE_FEATURES = "debug-tweaks eclipse-debug dev-pkgs"
dev-pkgs option adds -dev packages for all installed packages.
So now I have rootfs with all needed headers, and .so files pointing where they should.
Before compilation, I removed unnecessary Include path, leaving only the one from rootfs, and removed all the libraries, except pthread, and sndfile. But then I get new errors:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" -o "test.elf" ./src/main.o -lsndfile -lpthread
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find /lib/libpthread.so.0
makefile:48: polecenia dla obiektu 'test.elf' nie powiodły się (commands for ‘test.elf’ did not succeed)
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find /usr/lib/libpthread_nonshared.a
I spotted, that it looks for libraries in my root folder. Quick search in Google (and SO:)) told me that I should set –-sysroot variable. So I added it to Eclipse option (in Miscelenious card in Linker options) like that:
--sysroot=/home/stas/ZedboardPetalinuxFS
So now linker command looked like this:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" --sysroot=/home/stas/ZedboardPetalinuxFS -o "test.elf" ./src/main.o -lsndfile -lpthread
And all succeed! I also wrote simple example that uses pthreads, and sndfile, and it also worked. But WHY? This leads me to second question:
Why do I need --sysroot option in this case? When do I need to use this option in general? And why this time I didn't have to add all the dependencies to linking command?
3. Another idea
At this point, I had an idea, to check what will happen, if I add --sysroot option having rootfs populated with old, non development image. But this gave me new errors:
arm-linux-gnueabihf-g++ -L"/home/stas/ZedboardPetalinuxFS/usr/lib" -L"/home/stas/ZedboardPetalinuxFS/lib" --sysroot=/home/stas/ZedboardPetalinuxFS -o "test.elf" ./src/main.o -lpthread -lvorbisenc -lvorbis -logg -lFLAC -lsndfile
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find crt1.o: No such file or directory
makefile:48: polecenia dla obiektu 'test.elf' nie powiodły się
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find crti.o: No such file or directory
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lpthread
/opt/Xilinx/SDK/2016.4/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/5.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lm
So third question – what does this errors mean?
Thanks very much in advance!
"why I did not needed .so file for pthread, but needed it for all
other libraries?"
Actually you do need pthread.so file. You included pthread.h but didn't link with -lpthread. So it's normal you don't see any linker errors.
"when do I need .so file, and when .so.X file is enough"
When you give "-lNAME" parameter to g++, the compiler tells the linker to find libNAME.so within library search paths. Since there may exist multiple versions of the same library(libNAME.so.1, libNAME.so.1.20), *.so files link to desired actual library file. (Versioning of shared objects, ld man pages)
"Why do I need --sysroot option in this case? When do I need to use this option in general? And why this time I didn't have to add all the dependencies to linking command?"
The "dev-pkgs" in EXTRA_IMAGE_FEATURES changes your sysroot implicitly to let you link against the dev packages(yoctoproject image-features). That's why you need -sysroot option. You generally need this option when cross compiling to provide a root for standard search paths for headers and libraries. You didn't need it because you didn't have dev-pkgs image feature that changes your sysroot
"So third question – what does this errors mean?"
Even your the most basic hello world code gets linked with standard c library(if you didn't specify otherwise). libm.so, libpthread.so and crt1.o files are parts of libc library and come with libc dev package. So the linker can't see the standard library directories when it looks from your old sysroot
why I did not needed .so file for pthread, but needed it for all other libraries?
A cross compiler will normally come with a C Runtime (including pthread), typically in a directory that is part of the cross compiler installation.
The linker has built in search paths for libraries. These are in respect to the sysroot, which would by default be set to search the cross compiler's own included target C Runtime. If you added any -L options it would search those first and then move on to these pre-defined directories.
When you linked against pthread it would have found at least libpthread.a in the cross compiler's library directory.
Or more general – when do I need .so file, and when .so.X file is enough?
Shared libraries in Linux typically have a major and a minor version number. Libraries are ABI compatible between different minor versions with the same major version, but not between major versions. Sometimes there are three levels of versions but the principal is similar.
When installing libraries it is common to install the actual file with the full name, eg. libmy.so.1.2, then provide symlinks to libmy.so.1 and libmy.so.
If you are linking an application can work with any library version then you would just specify the name, eg. -lmy. In that case you would need symlinks from libmy.so to libmy.so.1.
If you required a specific version you would put -l:libmy.so.1. The ':' indicates a literal file name.
Linker scripts may affect things and may result in specific versions being selected even when you do specify the short name.
Why do I need --sysroot option in this case? When do I need to use
this option in general?
What --sysroot does is prepend the given path onto all the search directories which would normally be used to search for includes and libraries. It is most useful when cross compiling (as you are doing now) to get the compiler and the linker to search inside the target root instead of the build host's own root.
If you have specified a sysroot you probably do not need to specify include paths via -I or linker paths via -L, assuming that the files are within their normal spots inside your target root.
And why this time I didn't have to add all the dependencies to linking command?
One possible scenario is that the first time, sndfile for statically rather than dynamically linked. This would happen if your first root image had only sndfile.a in the lib dir, or elsewhere on the search path. To then satisfy the requirements of sndfile.a you would also need to link the other libs.
When linking against sndfile.so the dependencies will automatically get loaded via the dynamic linking process.
That's just a working theory at present.
So third question – what does this errors mean?
They mean it cannot find even the C runtime library to link.
As described for the first question, it was previously finding the C runtime in the pre-defined search path (relative to the predefined sysroot) which located the C runtime supplied by the cross compiler.
You disturbed this by supplying your own sysroot. It was now only searching the target root. Since this target root filesystem did not have development libs installed, there was no C runtime there to find.
You are doing several things wrong:
looks like you are not using environment variables, but calling cross-compiler directly. So, instead of compiling with arm-linux-gnueabihf-g++ ..., you should do $CXX .... The CXX is the environment variable set by the yocto script to set environment for cross compilation. Using CXX, you do not need to manually pass --sysroot
You should not link directly to pthread library with -lpthread. You should use -pthread
When I compile a c++ program using g++ from the command line and then do ldd a.out ldd is able to find libstdc++.a(libstdc++.so.6)
When I build a c++ ruby extension ldd myext.so cannot find libstdc++.a(libstdc++.so.6), and require 'myext' fails to load, with a complaint about not being able to find libstdc++.
If I run g++ -v I see the following output:
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/big_long_path....
Target: powerpc-ibm-aix7.1.0.0
Configured with: ../gcc-4.8.2/configure .....
Thread model: aix
gcc version 4.8.2 (GCC)
Now if I set my LIBPATH to include that big_long_path
export LIBPATH=/big_long_path....:$LIBPATH
ldd myext.so
is able to find libstdc++ and my require 'myext' works (returns true)
This might be OK, but I would rather not have to have users muck with their LIBPATH. Is there something I can add to my Makefile that allow the generated myext.so to find libstdc++ (and libgcc) in the location pointed to by that big_long_path I see in the COLLECT_LTO_WRAPPER line that I see when I run g++ -v ?
Update
The first link in the accepted answer below really helped me understand what was going on, and I was able to get ldd to not complain about libstdc++ by adding -blibpath:big_long_path:/usr:/usr/lib to the LDFLAGS in the Makefile.
But for some reason when ruby tried to load the ext it still failed. This made me think that ruby was somehow adjusting the LIBPATH. In the end my solution was to put a symbolic link to the libstdc++ and libgcc_s in the lib directory of the ruby install. The thought was ruby must need to search for extension shared objects, so I figured I would take advantage of this and place these two libs in the path that ruby must search. The only thing I am wondering is if I should just copy the libstdc++ and libgcc_s rather than symbolically link them?
It looks like you build your own gcc.
This is a known issue that gcc does not pass -rpath to the linker to specify the locations of libstdc++ and libgcc_s.
You either need to pass that path manually to the linker, or configure your own gcc to do that for you via specs file.
Don't let the loader searching for the libraries, use option -Wl,-bipath; check the result with dump -X32_64 -H; you should see something like this:
INDEX PATH BASE MEMBER
0 /usr/local/lib:/usr/lib:/lib
1 /usr/local/lib libapr-1.so.0
2 /usr/local/lib libaprutil-1.so.0
3 /usr/local/lib libcrypto.so.1.0.1f
4 /usr/local/lib libexpat.so.1
5 /usr/local/lib libgcc_s.a shr.o
6 /usr/local/lib libiconv.so.2
7 /usr/local/lib libssl.so.1.0.1f
8 /usr/local/lib libcpotlas.so.1
9 /usr/lib librtl.a shr.o
10 /usr/lib libc.a shr.o
11 .
Also I have to say using C++ for plugin is a really bad idea, especially in exotic systems like AIX
My LIBRARY_PATH environment variable has a custom directory in it: /cs/public/lib/pkg/opencv/lib.
But, when I use g++ --print-search-dirs, I get this instead:
libraries: =
/cs/public/lib/pkg/opencv/lib/x86_64-suse-linux/4.6/:
/cs/public/lib/pkg/opencv/lib/../lib64/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/../lib64/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/:
/lib/x86_64-suse-linux/4.6/:
/lib/../lib64/:
/usr/lib/x86_64-suse-linux/4.6/:
/usr/lib/../lib64/:
/cs/public/lib/pkg/opencv/lib/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../:
/lib/:
/usr/lib/
Why does g++ look in these alternatives and a whole bunch of other system locations before what I explicitly specify in the LIBRARY_PATH variable, and where is this documented?
I would understand if system defaults were searched before LIBRARY_PATH and LIBRARY_PATH/../lib64, etc, but g++ puts LIBRARY_PATH/../lib64, then system paths, then LIBRARY_PATH. Where is this ordering documented?
My g++ version is g++ (SUSE Linux) 4.6.2
My OS is openSUSE 12.1 (x86_64)
A similar question was asked here:
g++ searches /lib/../lib/, then /lib/
These scary-looking search paths are determined at least in part when the compiler itself it built, for example during the configure phase. It's clear that it goes beyond environment variables because it's possible to have multiple copies of GCC installed and have each of them give different results for gcc --print-search-dirs. Also noting that g++ --print-search-dirs and gcc --print-search-dirs give different results points out that the g++ wrapper is also affecting the search path. Besides configure/build time differences, GCC is definitely aware of the path where its own executable is, and will search subdirectories of that path. A lot of this alchemy can be found in the GCC documentation:
http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/Directory-Options.html#Directory-Options
http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/Environment-Variables.html#Environment-Variables
As far as I know, the most forceful thing that you can without compiling your own copy of GCC is to specify your custom libraries using the -L option. The reason I say this is because -L is searched before e.g. LIBRARY_PATH (see the above link on environment variables). In order to make it more tolerable you could add an alias for g++ including the -L option in your .bashrc file.
If you want a definitive answer then downloading a copy of the GCC source code is one way. For example, in gcc.c the following highly suggestive comment appears:
/* Build a list of search directories from PATHS.
PREFIX is a string to prepend to the list.
If CHECK_DIR_P is true we ensure the directory exists.
If DO_MULTI is true, multilib paths are output first, then
non-multilib paths.
This is used mostly by putenv_from_prefixes so we use `collect_obstack'.
It is also used by the --print-search-dirs flag. */
However the function that follows the comment is not very obvious.
This is multilib at work - a mechanism which allows for having libraries (but also the whole compilation and build toolchain) for multiple architectures on a single machine. This Wiki states that "The multilib suffix is appended to all directories searched for libraries by GCC and passed via -L options to the linker. The linker itself does not have any particular knowledge of multilibs, and will continue to consult its default search directories if a library is not found in the -L paths. If multiple orthogonal ABI-changing options are used in a single compilation, multiple multilib suffixes can be used in series.".
So, according to the above description, the architecture marker string or different variants thereof are appended to each library search path the compiler receives since it doesn't differentiate between default and custom paths. Your custom path is first in the row, but it undergoes the same "expansion" process as other paths.
Due to the need to handle i386 compatibility, multilib mechanisms seem to now be used by default on most x64 distros, which in practice means most of the installations out there.
I have the exact same problem on:
Fedora 17, gcc 4.7 and gcc 4.3
CentOS 6.3, gcc 4.4
Unubuntu 12, gcc 4.6
So it looks like this is a problem with most gcc versions. Probably this strange behavior first appeared in gcc 4.2 at least according to this.
I tried duping the specs and playing with them. It looks like the *multilib spec is used to append specific strings depending on the platform. For example my original spacs looked like:
*multilib:
. !m64 !m32;64:../lib64 m64 !m32;32:../lib !m64 m32;
When I changed 64:../lib64 to 64:../lib then instead of ../lib64 gcc appended ../lib. But I couldn't fully decipher the meaning of *multilib or any of the other specs.
This answer attempts to summarize the search path behaviors for both GCC and Clang.
GCC
Include path: for a command-line of the following format:
CPLUS_INCLUDE_PATH=EDIR g++ -IIDIR -isystemSDIR
the following list of directories are used as search paths for #include <...>:
IDIR # '-I' directories.
SDIR # '-isystem' directories.
EDIR # *_INCLUDE_PATH directories.
GCCDIR/include/c++/GCCVER # libstdc++ directory (C++).
GCCDIR/include/c++/GCCVER/GCCARCH # libstdc++ directory (C++).
GCCDIR/include/c++/GCCVER/backward # libstdc++ directory (C++).
GCCDIR/lib/gcc/GCCARCH/GCCVER/include # GCC arch-specific directory.
/usr/local/include/GCCARCH # Local arch-specific include directory.
/usr/local/include # Local include directory.
GCCDIR/include # GCC include directory.
GCCDIR/lib/gcc/GCCARCH/GCCVER/include-fixed # GCC include-fixed directory.
/usr/include/GCCARCH # System include arch-specific directory.
/usr/include # System include directory.
Library path: for a command-line of the following format:
LIBRARY_PATH=EDIR gcc -BBDIR -LLDIR
the following arguments are passed to the linker:
-LLDIR # '-L' directories.
-LBDIR # '-B' directories.
-LEDIR/../libXX # Multilib directories from LIBRARY_PATH.
-LGCCDIR/lib/gcc/GCCARCH/GCCVER # GCC arch-specific library directory.
-LGCCDIR/libXX # GCC multilib library directory.
-L/libXX # System multilib library directory.
-L/usr/libXX # System multilib library directory.
-LEDIR # LIBRARY_PATH directories.
-LGCCDIR/lib # Other GCC libraries.
Clang
Include path: for a command-line of the following format:
CPLUS_INCLUDE_PATH=EDIR clang++ --gcc-toolchain=GCCDIR -BBDIR -IIDIR -isystemSDIR
the following list of directories are used as search paths for #include <...>:
IDIR # '-I' directories.
SDIR # '-isystem' directories.
EDIR # *_INCLUDE_PATH directories.
# If -stdlib=libstdc++ is used:
GCCDIR/include/c++/GCCVER # libstdc++ directory from the selected GCC toolchain (C++).
GCCDIR/include/c++/GCCVER/GCCARCH # libstdc++ directory from the selected GCC toolchain (C++).
GCCDIR/include/c++/GCCVER/backward # libstdc++ directory from the selected GCC toolchain (C++).
# If -stdlib=libc++ is used:
CLANGDIR/include/c++/v1 # libc++ directory (C++).
/usr/local/include # Local include directory.
CLANGDIR/lib/clang/CLANGVER/include # Clang include directory.
/include # System include directory.
/usr/include # System include directory.
Library path: for a command-line of the following format:
LIBRARY_PATH=EDIR clang --gcc-toolchain=GCCDIR -BBDIR -LLDIR
the following arguments are passed to the linker:
-LLDIR # '-L' directories.
-LGCCDIR/lib/gcc/GCCARCH/GCCVER # GCC arch-specific library directory.
-LGCCDIR/libXX # GCC multilib library directory.
-L/libXX # System multilib library directory.
-L/usr/libXX # System multilib library directory.
-LGCCDIR/lib # Other GCC libraries.
-LCLANGDIR/lib # Clang libraries.
-L/lib # System library directory.
-L/usr/lib # System library directory.
-LEDIR # LIBRARY_PATH directories.
Summary
The search path for includes are pretty much the same in both GCC and Clang. C++-specific paths are omitted if the C frontend is used in both cases. Library search paths differ substantially between GCC and Clang, notably the presence of -B directories and the odd manipulation of LIBRARY_PATH in the GCC frontend.
The library search paths are the same for both C and C++ frontends. Other library search paths are introduced by the linker itself. The following excerpt comes from the vanilla linker script for GNU Binutils:
# Multilib library directories.
SEARCH_DIR("BINUTILSDIR/BINUTILSARCH/libXX");
SEARCH_DIR("BINUTILSDIR/libXX");
SEARCH_DIR("/usr/local/libXX");
SEARCH_DIR("/libXX");
SEARCH_DIR("/usr/libXX");
# Traditional library directories.
SEARCH_DIR("BINUTILSDIR/BINUTILSARCH/lib");
SEARCH_DIR("BINUTILSDIR/lib");
SEARCH_DIR("/usr/local/lib");
SEARCH_DIR("/lib");
SEARCH_DIR("/usr/lib");
It is also imperative to note that library dependencies are not searched for inside the directories listed above. These exclusively rely on -rpath and -rpath-link options passed to the linker, or else they get resolved from the default system library paths. Thus it might be useful to produce both -L and -rpath-link arguments in order to guarantee that the correct libraries are linked.
Finally, special files (such as the CRT objects) are searched for only in -B directories. In Clang, special files are also searched inside the selected GCC toolchain. Other factors (spec files, distribution-specific configuration) might change some or all of the above.
Looks like it's needed for cross-compiling. From the ChangeLog:
Wed Mar 29 14:53:23 1995 Jim Wilson <wilson#cygnus.com>
* gcc.c (process_command): Delete code modifying gcc_exec_prefix.
(main): Put it here after last use of gcc_exec_prefix. For cross
compiler, set startfile_prefixes if gcc_exec_prefix is set and
standard_startfile_prefix is a relative path.
startfile_prefixes is what is being printed out with the search-dirs flags. From gcc/gcc.c:
if (print_search_dirs)
{
printf (_("install: %s%s\n"), standard_exec_prefix, machine_suffix);
printf (_("programs: %s\n"), build_search_list (&exec_prefixes, "", 0));
printf (_("libraries: %s\n"), build_search_list (&startfile_prefixes, "", 0));
return (0);
}
The compiler will first look at the default paths then for the others.
How does it sorted when you print I don't now, but it is documented here, 3.19 Environment Variables Affecting GCC.
The paths are defined by the built-in specs. Specs define how the pipeline processes the source code to obtain a result. GCC just drives the compilation.
You can give GCC your own spec file via -spec=, and you can get the built-in specs with -dumpspecs IIRC.
This is probably explained somewhere in the GCC manual.