Aim Compile a C++ program on Windows for ARM using only LLVM.
Why LLVM because of permissive licensing.
I'm starting to wonder if my understanding of LLVM is correct.
On the host machine do
Use clang (front end) to generate intermediate representation. This representation is target independent.
Use llc (back end) to generate target assembly code.
Use lld-link.exe to produce executable.
Then execute on the target machine.
Host machine Windows 10, 64bit
Target machine Drive PX with a arm cortex-a57
The program
int main(int argc, char* argv[])
{
int x=41;
x++;
return x;
}
I've checked out and compiled LLVM (using Visual Studio 2015, Release build, CPU= x64)
My attempts
clang.exe -target arm -march=armv8-a -mcpu=cortex-a57 -mfloat-abi=hard -emit-llvm -c -o main.bc main.cpp
llc.exe -march=arm -mcpu=cortex-a57 -mattr=a57,armv8-a,v8 -meabi=gnu -o main.s main.bc
lld-link.exe /entry:main /machine:arm main.s
Error
lld-link.exe: error: main.s: unknown file type
Then I tried doing the front-end steps on Windows and the back-end on the arm machine.
clang.exe -target arm -march=armv8-a -mcpu=cortex-a57 -mfloat-abi=hard -emit-llvm -c -o main.bc main.cpp
llc.exe -march=arm -mcpu=cortex-a57 -mattr=a57,armv8-a,v8 -meabi=gnu -o main.s main.bc
SCP main.s to the arm machine. SSH and
gcc main.s (using gcc as a test. LLVM should do this.)
Error
main.s: Assembler messages:
main.s:2: Error: unknown pseudo-op: `.syntax'
main.s:3: Error: unknown pseudo-op: `.eabi_attribute'
main.s:9: Error: unknown pseudo-op: `.fpu'
main.s:26: Error: junk at end of line, first unrecognized character is `#'
main.s:29: Error: unknown pseudo-op: `.code'
main.s:31: Error: unknown pseudo-op: `.fnstart'
main.s:32: Error: junk at end of line, first unrecognized character is `#'
main.s:34: Error: operand 1 should be an integer register -- `mov r2,#0'
main.s:41: Error: operand 1 should be an integer or stack pointer register -- `add r0,r0,#1'
main.s:45: Error: unknown mnemonic `bx' -- `bx lr'
main.s:48: Error: unknown pseudo-op: `.cantunwind'
main.s:49: Error: unknown pseudo-op: `.fnend'
main.s:50: Error: junk at end of line, first unrecognized character is `#'
So I tried to target only Windows
clang.exe -emit-llvm -c -o main.bc main.cpp
llc.exe -march=x86 -c -o main.s main.bc
ld.lld.exe main.s
Error
ld.lld.exe: error: main.s:1: unknown directive: .text
Then, instead of ld.lld.exe use gcc (Again using gcc as a test. LLVM should do this.)
clang.exe -emit-llvm -c -o main.bc main.cpp
llc.exe -march=x86 -c -o main.s main.bc
gcc main.s -o main.exe
That works. To test I type
main.exe
echo Exit Code is %errorlevel%
Which returns 42
General question
What are the steps to compile a C++ program under Windows targeting an arm CPU using only LLVM (no gcc, nothing downloaded from ARM)?
Specific questions
Can the tools that come with self-compiled LLVM (e.g. clang.exe, llc.exe, lld.exe) compile an executable on Windows targeting arm? E.g is lld still under development?
Why does my attempt to compile and link under Windows, targeting Windows fail?
Where do the header files and libraries (e.g. libstdc++) come from when linking on the host for the target? I suppose I need to get those from the arm machine? Copy them to the host and tell the linker where to find them ? Is that correct?
Update
So I originally tried Cross-compilation using Clang
clang.exe --target=arm --sysroot=c:\code\clang\FromCmdLine main.cpp -v
The result is
clang.exe: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
And the details of -v are
"C:\\llvm\\clang.exe" -cc1 -triple armv4t-- -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu arm7tdmi -target-feature +soft-float -target-feature +soft-float-abi -target-feature -fp-only-sp -target-feature -d16 -target-feature -vfp2 -target-feature -vfp3 -target-feature -fp16 -target-feature -vfp4 -target-feature -fp-armv8 -target-feature -neon -target-feature -crypto -target-feature +strict-align -target-abi aapcs -msoft-float -mfloat-abi soft -fallow-half-arguments-and-returns -dwarf-column-info -debugger-tuning=gdb -v -resource-dir "c:\\llvm\\clang\\7.0.0" -isysroot "c:\\code" -fdeprecated-macro -fdebug-compilation-dir "c:\\code" -ferror-limit 19 -fmessage-length 293 -fno-signed-char -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\AppData\\Local\\Temp\\main-b17d06.o" -x c++ main.cpp
clang -cc1 version 7.0.0 based upon LLVM 7.0.0svn default target x86_64-pc-win32
ignoring nonexistent directory "c:\code\usr/local/include"
ignoring nonexistent directory "c:\code\usr/include"
#include "..." search starts here:
#include <...> search starts here:
C:\llvm\clang\7.0.0\include
End of search list.
"C:\\MinGW\\bin\\gcc.exe" "--sysroot=c:\\code" -v -o a.out "C:\\Users\\AppData\\Local\\Temp\\main-b17d06.o"
Using built-in specs.
COLLECT_GCC=C:\MinGW\bin\gcc.exe
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/lto-wrapper.exe
Target: mingw32
Configured with: ../src/gcc-6.3.0/configure --build=x86_64-pc-linux-gnu --host=mingw32 --target=mingw32 --with-gmp=/mingw --with-mpfr --with-mpc=/mingw --with-isl=/mingw --prefix=/mingw --disable-win32-registry --with-arch=i586 --with-tune=generic --enable-languages=c,c++,objc,obj-c++,fortran,ada --with-pkgversion='MinGW.org GCC-6.3.0-1' --enable-static --enable-shared --enable-threads --with-dwarf2 --disable-sjlj-exceptions --enable-version-specific-runtime-libs --with-libiconv-prefix=/mingw --with-libintl-prefix=/mingw --enable-libstdcxx-debug --enable-libgomp --disable-libvtv --enable-nls
Thread model: win32
gcc version 6.3.0 (MinGW.org GCC-6.3.0-1)
COMPILER_PATH=c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/;c:/mingw/bin/../libexec/gcc/;c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/bin/
LIBRARY_PATH=c:/mingw/bin/../lib/gcc/mingw32/6.3.0/;c:/mingw/bin/../lib/gcc/;c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/lib/;c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../;c:/code/clang/FromCmdLine/lib/
COLLECT_GCC_OPTIONS='-v' '-o' 'a.out' '-mtune=generic' '-march=i586'
c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/collect2.exe -plugin c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/liblto_plugin-0.dll -plugin-opt=c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/lto-wrapper.exe -plugin-opt=-fresolution=C:\Users\AppData\Local\Temp\ccufvVIA.res -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-ladvapi32 -plugin-opt=-pass-through=-lshell32 -plugin-opt=-pass-through=-luser32 -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt --sysroot=c:\code\clang\FromCmdLine -Bdynamic -o a.out c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../crt2.o c:/mingw/bin/../lib/gcc/mingw32/6.3.0/crtbegin.o -Lc:/mingw/bin/../lib/gcc/mingw32/6.3.0 -Lc:/mingw/bin/../lib/gcc -Lc:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/lib -Lc:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../.. -Lc:/code/clang/FromCmdLine/lib C:\Users\AppData\Local\Temp\main-b17d06.o -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt c:/mingw/bin/../lib/gcc/mingw32/6.3.0/crtend.o
c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/bin/ld.exe: C:\Users\AppData\Local\Temp\main-b17d06.o: Relocations in generic ELF (EM: 40)
C:\Users\AppData\Local\Temp\main-b17d06.o: error adding symbols: File in wrong format
Update
This does not fully answer my question but it does help me to progress.
For a better understanding I found crosstool-NG useful, especially their documentation (chapters 1 to 5).
Then I read the cmake cross compiling documentation.
The I wrote a small cmake C++ test.
Helloworld.cpp
#include <iostream>
int main(int argc, char *argv[])
{
std::cout << "Hello World!" << std::endl;
return 0;
}
CMakeLists.txt
cmake_minimum_required(VERSION 2.8.9)
project (hello)
add_executable(hello helloworld.cpp)
Target specific configuration for cmake. This is from 4.
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)
set(CMAKE_SYSROOT /home/user/x-tools/aarch64-unknown-linux-gnueabi/aarch64-unknown-linux-gnueabi/sysroot/)
set(CMAKE_STAGING_PREFIX /home/user/crosscompile/stage)
set(tools /home/user/x-tools/aarch64-unknown-linux-gnueabi)
set(CMAKE_C_COMPILER ${tools}/bin/aarch64-unknown-linux-gnueabi-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/aarch64-unknown-linux-gnueabi-g++)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
And the command line
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchain_file.txt ..
That cross compiles to ARM and the program runs on the ARM machine.
But this does not use LLVM / Clang. To use LLVM I thought of changing the toolchain configuration to use
set(tools /usr/bin)
set(CMAKE_C_COMPILER ${tools}/clang)
set(CMAKE_CXX_COMPILER ${tools}/clang++)
That failed because that bin folder is for the host machine.
I also tried using the AArch64 download from http://releases.llvm.org/download.html. Yes that also did not work.
So in summary this what is required.
A sysroot folder with the lib and include folders for the target system. Okay there needs to be more in that sysroot folder than lib and include.
A toolchain (compiler, assembler, linker) for the target system.
This is what I had to do, to get a proof-of-concept working, to cross compile using only llvm, with host=linux x86_64 and target = DrivePX (arm aarch64). (Also worked with host=Windows 10 x86_64.)
I recommend a tool like croostool-ng which sets up a cross compilation toolchain for you, but the steps below show what's going on behind the scenes, and it uses only llvm.
On the host computer, checkout and compile LLVM including Clang
see http://llvm.org/docs/GettingStarted.html.
This takes a long time so do a release build. Also make sure you have plenty of space (GB).
To tell cmake to do a release build e.g.
CC=gcc CXX=g++ cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release ../llvm/
Then call make.
Call clang on the host computer.
Remember to set the target, e.g. --target=aarch64-linux-gnu
Tell clang to use the llvm linker, i.e. -fuse-ld=lld
clang --target=aarch64-linux-gnu -v main.cpp -o main -fuse-ld=lld
There will be a lot of linker errors
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: unable to find library -lc
And some more errors about missing crtX.o files, e.g.
ld.lld: error: cannot open crt1.o: No such file or directory
On the target machine find, all the libgcc, libc, etc. files.
Remember to copy the fully versioned libraries, e.g. for libgcc_s.so copy libgcc_s.so.1
Copy these files to the host computer.
Tell clang where to find these libraries, e.g. Copy the files into a folder called libs
clang --target=aarch64-linux-gnu -v main.cpp -o main -fuse-ld=lld -L./libs
Also remember that the linker is looking for e.g. libgcc_s.so so create symbolic links to e.g. libgcc_s.so.1.
Copy the crtX.o files from the target to the host. Put them next to the main.cpp.
Then one more error
ld.lld: error: undefined symbol: __libc_csu_fini
That symbol is located in libc_nonshared.a. And this answer helped.
libc.so is a linker script which tells the linker to pull in the shared library libc.so.6, and a non-shared portion, libc_nonshared.a
On the host find and open libc.so and see where libc.so.6 and libc_nonshared.a are located.
Copy them to host.
Tell the linker to use those 2 libraries, i.e. -lc -lc_nonshared
clang --target=aarch64-linux-gnu -v main.cpp -o main -fuse-ld=lld -L./libs -lc -lc_nonshared
My short text code only depends on libc, libgcc and requires no header files. If your code requires other libraries and header files you would have to copy them from the target to the host.
Update
Read this question if you are wondering about libgcc.
Related
I wanted to try and use clang instead of gcc on my Ubuntu system. I set up the most basic C++ project possible, with a minimal CMakeLists.txt:
cmake_minimum_required(VERSION 3.20)
project(pcb_engine)
add_executable(main main.cpp)
My main.cpp is also very simple:
#include <iostream>
int main() {
std::cout << "Hello World" << std::endl;
return 0;
}
No when I run g++ -o main_g++ ../main.cpp everything compiles just fine. Also, if I set my system's default compiler to be gcc and g++ using
sudo update-alternatives --config c++
sudo update-alternatives --config cc
and I run cmake .. and make from my build folder, everything works!
The error:
Now I switch to using clang and clang++ as my default compiler via the update-alternatives command and while the build file generation succeeds, the make command fails with the following error message:
fatal error: 'iostream' file not found
#include <iostream>
When I run clang++ -v -stdlib=libc++ -Wall ../main.cpp -o main_clang manually, I get a linker error:
Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
Candidate multilib: .;#m64
Selected multilib: .;#m64
"/usr/lib/llvm-14/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/ivan/Documents/TEST/build -resource-dir /usr/lib/llvm-14/lib/clang/14.0.0 -internal-isystem /usr/lib/llvm-14/bin/../include/c++/v1 -internal-isystem /usr/lib/llvm-14/lib/clang/14.0.0/include -internal-isystem /usr/local/include -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -fdeprecated-macro -fdebug-compilation-dir=/home/ivan/Documents/TEST/build -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-482542.o -x c++ ../main.cpp
clang -cc1 version 14.0.0 based upon LLVM 14.0.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/lib/llvm-14/bin/../include/c++/v1
/usr/lib/llvm-14/lib/clang/14.0.0/include
/usr/local/include
/usr/include/x86_64-linux-gnu
/usr/include
End of search list.
"/usr/bin/ld" -pie -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o main_clang /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/12/crtbeginS.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/12 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/lib/llvm-14/bin/../lib -L/lib -L/usr/lib /tmp/main-482542.o -lc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/bin/../lib/gcc/x86_64-linux-gnu/12/crtendS.o /lib/x86_64-linux-gnu/crtn.o
/usr/bin/ld: /tmp/main-482542.o: undefined reference to symbol '__gxx_personality_v0'
/usr/bin/ld: /lib/x86_64-linux-gnu/libc++abi.so.1: error adding symbols: DSO missing from command line
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The non-verbose error message:
/usr/bin/ld: /tmp/main-f39fec.o: undefined reference to symbol '__gxx_personality_v0'
/usr/bin/ld: /lib/x86_64-linux-gnu/libc++abi.so.1: error adding symbols: DSO missing from command line
Can anyone help me identify where to start fixing this?
Interestingly, when I run clang++ -v -stdlib=libstdc++ -Wall ../main.cpp -o main_clang (note the -stdlib=libstdc++ flag change to the gnu standard library), then I get the same simple fatal error: 'iostream' file not found error as with cmake! So there is a difference in the behavior with the two different standard c++ libraries...
The background:
I come from using gcc, and installed clang via get-apt install clang. I've read that clang does not come with its own standard C++ library, but I have trouble understanding the implications for me: Previously, I had issues with clang finding a correct standard library .so file... I tried some more installs and I have got the libstdc++.so (default gnu) and the libc++.so as well as libc++.a and libc++.abi.so in my /usr/lib/x86_64-linux-gnu folder with a correct symlink hierarchy. For some reason, after a system restart, the /usr/bin/ld: cannot find -lstdc++ error went away, and now I have this linker error where it seems that clang can not find the include header files when running cmake. Now, I am at the end of my limited wisdom.
I am attempting to cross-compile a C++ object file using the GNU ARM Embedded Toolchain libraries, and the Clang compiler. I cannot understand what is missing to successfully compile. Could someone help me to understand why clang cannot find the header files?
Here is my example program:
#include <algorithm>
extern "C" uint32_t * copy(
uint32_t const * first, uint32_t const * last, uint32_t * dest)
{
return std::copy(first, last, dest);
}
I invoke clang++ by specifying the target and sysroot:
$ clang++ --target=thumbv7m-none-eabi --sysroot "C:\Program Files (x86)\GNU Arm Embedded Toolchain\10 2021.07\arm-none-eabi" -c example.cpp -v
clang version 12.0.1
Target: thumbv7m-none-unknown-eabi
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
(in-process)
"C:\\Program Files\\LLVM\\bin\\clang++.exe" -cc1 -triple thumbv7m-none-unknown-eabi -emit-obj -mrelax-all --mrelax-relocations -disable-free -disable-llvm-verifier -discard-value-names -main-file-name example.cpp -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -fno-verbose-asm -mconstructor-aliases -nostdsysteminc -target-cpu cortex-m3 -target-feature +soft-float-abi -target-feature +strict-align -target-abi aapcs -mfloat-abi soft -fallow-half-arguments-and-returns -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir "C:\\Program Files\\LLVM\\lib\\clang\\12.0.1" -isysroot "C:\\Program Files (x86)\\GNU Arm Embedded Toolchain\\10 2021.07\\arm-none-eabi" -internal-isystem "C:\\Program Files (x86)\\GNU Arm Embedded Toolchain\\10 2021.07\\arm-none-eabi\\include\\c++\\v1" -internal-isystem "C:\\Program Files\\LLVM\\lib\\clang\\12.0.1\\include" -internal-isystem "C:\\Program Files (x86)\\GNU Arm Embedded Toolchain\\10 2021.07\\arm-none-eabi\\include" -fdeprecated-macro -fdebug-compilation-dir "C:\\source\\espresso\\components" -ferror-limit 19 -fmessage-length=226 -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -o example.o -x c++ example.cpp
clang -cc1 version 12.0.1 based upon LLVM 12.0.1 default target x86_64-pc-windows-msvc
ignoring nonexistent directory "C:\Program Files (x86)\GNU Arm Embedded Toolchain\10 2021.07\arm-none-eabi\include\c++\v1"
ignoring duplicate directory "C:\Program Files\LLVM\lib\clang\12.0.1\include"
#include "..." search starts here:
#include <...> search starts here:
C:\Program Files\LLVM\lib\clang\12.0.1\include
C:\Program Files (x86)\GNU Arm Embedded Toolchain\10 2021.07\arm-none-eabi\include
End of search list.
example.cpp:3:10: fatal error: 'algorithm' file not found
#include <algorithm>
^~~~~~~~~~~
1 error generated.
I would suspect a linker error (clang may not know how to find the correct libraries within the sysroot), however I am surprised clang cannot infer the location of the algorithm header given a sysroot.
Any suggestions?
Both clang and arm-none-eabi-gcc are in my path:
$ clang++ --version
clang version 12.0.1
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
$ arm-none-eabi-gcc --version
arm-none-eabi-gcc (GNU Arm Embedded Toolchain 10.3-2021.07) 10.3.1 20210621 (release)
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
I've found that -sysroot doesn't solve the problem on my system and so - as suggested by the relevant Clang documentation - I've explicitly specified include paths using -isystem:
ifeq ($(toolchain),clang)
INCLUDE += -isystem "$(GCC_TOOLCHAIN_DIR)/arm-none-eabi/include/c++/$(GCC_TOOLCHAIN_VERSION)/arm-none-eabi"
INCLUDE += -isystem "$(GCC_TOOLCHAIN_DIR)/arm-none-eabi/include/c++/$(GCC_TOOLCHAIN_VERSION)"
INCLUDE += -isystem "$(GCC_TOOLCHAIN_DIR)/arm-none-eabi/include"
endif
Since the GCC header files use #include_next directives, the order of these is crucial.
For the time being I'm using gcc/g++ to link, so I'm yet to have to deal with any linking issues.
That's the question. Basically my problem is that I am trying to compile an app for ARM with an options -fsanitize=address -static-libasan (cool thing for debugging a wrong pointers and etc.). This compiles fine directly to executable file, but doesn't work for compiling from an object files. Ofc I found that the static lib is libasan.a; anyway when I am trying to link everything from an object file, it says about a bunch of an undefined references in «pthread» and some functions whose lib-owner I didn't even managed to find.
So, I just want to compile a file with -fsanitize=address -static-libasan and see which a libraries were linked statically, so that I could later do it from an object file.
An executable stores no trace of any static libraries it pulled object files from. Mostly these libraries are explicitly specified when linking, though. The compiler only pulls in a few standard static libraries. For gcc you can see various information including the actual command line passed to the linker using the -v option (it seems some implementations want to use --verbose and for the linker -Wl,--verbose). This output should show all implicitly linked libraries like -lgcc and -lstdc++.
Note that you should link using the language specific frontend and the same compiler flags as when compiling. Your problem description sounds as if you use -pthread with g++ when compiling but you tried to link using gcc without the -pthread flags (there are a few different threading related flags; from the description it is impossible to tell which one was used).
Just to deal with the insults, here is an example output when using the -v option on my computer:
ThreadGard:stackoverflow kuehl$ cat hello.cpp
#include <iostream>
int main()
{
std::cout << "hello, world\n";
return 0;
}
ThreadGard:stackoverflow kuehl$ g++ -v -static hello.cpp
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/opt/gcc-4.9.2/libexec/gcc/x86_64-apple-darwin13.4.0/4.9.2/lto-wrapper
Target: x86_64-apple-darwin13.4.0
Configured with: ../gcc-4.9.2/configure --prefix=/opt/gcc-4.9.2 --with-gmp=/opt/gcc-infrastructure --with-mpfr=/opt/gcc-infrastructure --enable-decimal-float=bid --enable-lto --enable-languages=c,c++
Thread model: posix
gcc version 4.9.2 (GCC)
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.4' '-v' '-static' '-mtune=core2'
/opt/gcc-4.9.2/libexec/gcc/x86_64-apple-darwin13.4.0/4.9.2/cc1plus -quiet -v -D__STATIC__ hello.cpp -quiet -dumpbase hello.cpp -mmacosx-version-min=10.9.4 -mtune=core2 -auxbase hello -version -o /var/folders/b_/64plfvs936v5ylx36qwc8rg00000gp/T//ccrt7fXS.s
GNU C++ (GCC) version 4.9.2 (x86_64-apple-darwin13.4.0)
compiled by GNU C version 4.9.2, GMP version 5.0.5, MPFR version 3.1.1, MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../../x86_64-apple-darwin13.4.0/include"
#include "..." search starts here:
#include <...> search starts here:
/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../../include/c++/4.9.2
/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../../include/c++/4.9.2/x86_64-apple-darwin13.4.0
/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../../include/c++/4.9.2/backward
/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/include
/usr/local/include
/opt/gcc-4.9.2/include
/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/include-fixed
/usr/include
/System/Library/Frameworks
/Library/Frameworks
End of search list.
GNU C++ (GCC) version 4.9.2 (x86_64-apple-darwin13.4.0)
compiled by GNU C version 4.9.2, GMP version 5.0.5, MPFR version 3.1.1, MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: fea0def5b69cf8ae349b2f1faf4b1d23
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.4' '-v' '-static' '-mtune=core2'
as -arch x86_64 -force_cpusubtype_ALL -static -o /var/folders/b_/64plfvs936v5ylx36qwc8rg00000gp/T//ccPF77oR.o /var/folders/b_/64plfvs936v5ylx36qwc8rg00000gp/T//ccrt7fXS.s
COMPILER_PATH=/opt/gcc-4.9.2/libexec/gcc/x86_64-apple-darwin13.4.0/4.9.2/:/opt/gcc-4.9.2/libexec/gcc/x86_64-apple-darwin13.4.0/4.9.2/:/opt/gcc-4.9.2/libexec/gcc/x86_64-apple-darwin13.4.0/:/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/:/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/
LIBRARY_PATH=/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/:/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../:/usr/lib/
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.4' '-v' '-static' '-mtune=core2'
/opt/gcc-4.9.2/libexec/gcc/x86_64-apple-darwin13.4.0/4.9.2/collect2 -static -arch x86_64 -macosx_version_min 10.9.4 -weak_reference_mismatches non-weak -o a.out -lcrt0.o -L/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2 -L/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../.. /var/folders/b_/64plfvs936v5ylx36qwc8rg00000gp/T//ccPF77oR.o /opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../libstdc++.a -no_compact_unwind -no_pie -lgcc_eh -lgcc -v
collect2 version 4.9.2
/usr/bin/ld -static -arch x86_64 -macosx_version_min 10.9.4 -weak_reference_mismatches non-weak -o a.out -lcrt0.o -L/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2 -L/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../.. /var/folders/b_/64plfvs936v5ylx36qwc8rg00000gp/T//ccPF77oR.o /opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../libstdc++.a -no_compact_unwind -no_pie -lgcc_eh -lgcc -v
#(#)PROGRAM:ld PROJECT:ld64-241.9
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7m armv7em
Library search paths:
/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2
/opt/gcc-4.9.2/lib
/usr/lib
/usr/local/lib
Framework search paths:
/Library/Frameworks/
/System/Library/Frameworks/
ld: library not found for -lcrt0.o
collect2: error: ld returned 1 exit status
There are not all static libraries installed to make it actually link. However, that's besides the point as the output clearly lists the linker invocation:
/usr/bin/ld -static -arch x86_64 -macosx_version_min 10.9.4 -weak_reference_mismatches non-weak -o a.out -lcrt0.o -L/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2 -L/opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../.. /var/folders/b_/64plfvs936v5ylx36qwc8rg00000gp/T//ccPF77oR.o /opt/gcc-4.9.2/lib/gcc/x86_64-apple-darwin13.4.0/4.9.2/../../../libstdc++.a -no_compact_unwind -no_pie -lgcc_eh -lgcc -v
(with some emphasis added to make it easier for readers - it seems to be necessary)
I've been trying to figure out why g++ cannot link a program with the armadillo library. The problem is simple:
macbook-pro:arma-xc jmlopez$ g++-4.9 inputs-arma.cpp -larmadillo
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status
Perhaps I did specify where the library should be
macbook-pro:arma-xc jmlopez$ ls -l /usr/lib/libarma*
-rwxr-xr-x 1 root wheel 37928 Aug 18 14:30 /usr/lib/libarmadillo.4.32.2.dylib
lrwxr-xr-x 1 root wheel 25 Aug 18 14:30 /usr/lib/libarmadillo.4.dylib -> libarmadillo.4.32.2.dylib
lrwxr-xr-x 1 root wheel 20 Aug 18 14:30 /usr/lib/libarmadillo.dylib -> libarmadillo.4.dylib
Let us try again
macbook-pro:arma-xc jmlopez$ g++-4.9 -L/usr/lib inputs-arma.cpp -larmadillo
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status
Ok, that did not fix it. Let us see what is going on, for this we will use the -v option which will generate some long output.
macbook-pro:arma-xc jmlopez$ g++-4.9 -L/usr/lib inputs-arma.cpp -larmadillo -v
Using built-in specs.
COLLECT_GCC=g++-4.9
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/lto-wrapper
Target: x86_64-apple-darwin13.3.0
Configured with: ../configure --build=x86_64-apple-darwin13.3.0 --prefix=/usr/local/Cellar/gcc/4.9.1 --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-4.9 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-cloog=/usr/local/opt/cloog --with-isl=/usr/local/opt/isl --with-system-zlib --enable-version-specific-runtime-libs --enable-libstdcxx-time=yes --enable-stage1-checking --enable-checking=release --enable-lto --disable-werror --with-pkgversion='Homebrew gcc 4.9.1' --with-bugurl=https://github.com/Homebrew/homebrew/issues --enable-plugin --disable-nls --enable-multilib --with-native-system-header-dir=/usr/include --with-sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk
Thread model: posix
gcc version 4.9.1 (Homebrew gcc 4.9.1)
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.3' '-L/usr/lib' '-v' '-shared-libgcc' '-mtune=core2'
/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/cc1plus -quiet -v -D__DYNAMIC__ inputs-arma.cpp -fPIC -quiet -dumpbase inputs-arma.cpp -mmacosx-version-min=10.9.3 -mtune=core2 -auxbase inputs-arma -version -o /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//ccApA95Q.s
GNU C++ (Homebrew gcc 4.9.1) version 4.9.1 (x86_64-apple-darwin13.3.0)
compiled by GNU C version 4.9.1, GMP version 6.0.0, MPFR version 3.1.2-p8, MPC version 1.0.2
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/local/include"
ignoring nonexistent directory "/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../../../x86_64-apple-darwin13.3.0/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/Users/jmlopez/github/excentury/excentury/extern/include
/usr/include
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include/c++
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include/c++/x86_64-apple-darwin13.3.0
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include/c++/backward
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include-fixed
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks
End of search list.
GNU C++ (Homebrew gcc 4.9.1) version 4.9.1 (x86_64-apple-darwin13.3.0)
compiled by GNU C version 4.9.1, GMP version 6.0.0, MPFR version 3.1.2-p8, MPC version 1.0.2
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 5a051bcf2be886a5c6eb8ba33f338693
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.3' '-L/usr/lib' '-v' '-shared-libgcc' '-mtune=core2'
as -arch x86_64 -force_cpusubtype_ALL -o /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//cchm9rVF.o /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//ccApA95Q.s
COMPILER_PATH=/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/
LIBRARY_PATH=/usr/lib/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../../:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib/
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.3' '-L/usr/lib' '-v' '-shared-libgcc' '-mtune=core2'
/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/collect2 -dynamic -arch x86_64 -macosx_version_min 10.9.3 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -weak_reference_mismatches non-weak -o a.out -L/usr/lib -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1 -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../.. -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//cchm9rVF.o -larmadillo -lstdc++ -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
collect2 version 4.9.1
/usr/bin/ld -dynamic -arch x86_64 -macosx_version_min 10.9.3 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -weak_reference_mismatches non-weak -o a.out -L/usr/lib -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1 -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../.. -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//cchm9rVF.o -larmadillo -lstdc++ -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
#(#)PROGRAM:ld PROJECT:ld64-236.4
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 armv6m armv7m armv7em
Library search paths:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
/usr/local/Cellar/gcc/4.9.1/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
Framework search paths:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status
The important information is here:
Library search paths:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
/usr/local/Cellar/gcc/4.9.1/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
Somehow it is not looking in /usr/lib. I have tried specifying this directory by first
exporting the variable LD_LIBRARY_PATH
macbook-pro:arma-xc jmlopez$ export LD_LIBRARY_PATH=/usr/lib
macbook-pro:arma-xc jmlopez$ g++-4.9 -L/usr/lib inputs-arma.cpp -larmadillo
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status
No luck. Finally, thinking that ld might be broken I did this:
macbook-pro:arma-xc jmlopez$ ln -s /usr/lib/libarmadillo.dylib /usr/local/Cellar/gcc/4.9.1/lib/libarmadillo.dylib
macbook-pro:arma-xc jmlopez$ g++-4.9 inputs-arma.cpp -larmadillo
Now it works. I'm not sure if there is something wrong with my system but it seems that I have no way of specifying other paths for ld to look for my libraries. Instead I have to use any of the following:
Library search paths:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
/usr/local/Cellar/gcc/4.9.1/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
Is there a way to fix this? Notice that I'm using the g++ from homebrew since I thought that the one that came with xcode was broken. In any case, no luck, at least the homebrew g++ told me the paths where the library was being searched.
Update:
I decided to see if I could add another path:
g++-4.9 -L/Users/jmlopez/ -L/Users/jmlopez/Desktop -L/usr/lib inputs-arma.cpp -larmadillo -v
Now the list of paths where it searches for the libraries is:
Library search paths:
/Users/jmlopez/
/Users/jmlopez/Desktop
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
/usr/local/Cellar/gcc/4.9.1/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
Where is /usr/lib? What is so special about this directory that ld refuses to look in this directory. It searches in my home directory and in my Desktop. I tried adding /usr/local and it adds it to the list of directories. I'm now starting to wonder about the permissions on the directories... Any ideas?
It is likely that your homebrew g++ was compiled with a custom prefix. The prefix for gcc is usually /usr, meaning that it will look for binaries in /usr/bin, headers in /usr/include and libraries in /usr/lib. If you've compiled with a custom prefix (/usr/local/Cellar/gcc/4.9.1/ in this case, it seems) the it won't look in /usr/lib.
LD_LIBRARY_PATH is how you tell the runtime linker where to look for libraries. So if you had linked to /usr/my/bizarre/lib/path/libmylib.so, then you run it like this:
> LD_LIBRARY_PATH=/usr/my/bizarre/lib/path myprog
To tell g++ where to find libraries, you use the -L command-line option. So your command-line should look like this:
g++-4.9 inputs-arma.cpp -L/usr/lib -larmadillo
I ran into this thread trying to answer a similar question myself. Hopefully this will help out.
You need ld_library_path for the homebrew compiler to work and dyld_library_path for os x to appropriately use the library.
put the following exports somewhere where they can get sourced.
export LD_LIBRARY_PATH=/usr/lib:$LD_LIBRARY_PATH
export DYLD_LIBRARY_PATH=$LD_LIBRARY_PATH:$DYLD_LIBRARY_PATH
This worked for me when compiling with homebrew gcc on OS X 10.9 and 10.10.
The reason for this is that Xcode sets the SDK (in your case the macOS SDK) as the virtual root. So all search paths of the compiler and the linker are being interpreted as relative to the SDK folder.
Specifically, the Linker will not look in /usr/lib, it will look in [macOS SDK]/usr/lib.
If you need to use /usr/lib (e.g. when linking against OpenSSL's libssl which only exist in /usr/lib but not in [macOS SDK]/usr/lib, you can set "-isysroot /" as the first two "Other Linker Flags" of your project's or target's build settings. This ensures that the Linker will use / as the root (and hence will find libraries in /usr/lib), while the compiler will continue to use the virtual root set by Xcode (i.e. the macOS SDK).
Just compiled clang-2.9 release and it can't link an hello world example. There is the error:
crt1.o: No such file: No such file or directory
LLVM is configured as default + --enable-shared. The llvm-2.8 build with same options works normally on the same machine.
$ clang -v a.c
clang version 2.9 (tags/RELEASE_29/final)
Target: i386-pc-linux-gnu
Thread model: posix
"/root/bin/clang" -cc1 -triple i386-pc-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -main-file-name a.c -mrelocation-model static -mdisable-fp-elim -masm-verbose -mconstructor-aliases -target-cpu pentium4 -target-linker-version 2.18 -momit-leaf-frame-pointer -v -resource-dir /root/lib/clang/2.9 -ferror-limit 19 -fmessage-length 157 -fgnu-runtime -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/cc-2ueoYy.o -x c a.c
clang -cc1 version 2.9 based upon llvm 2.9 hosted on i386-pc-linux-gnu
ignoring nonexistent directory "/usr/local/include"
#include "..." search starts here:
#include <...> search starts here:
/root/lib/clang/2.9/include
/usr/include
End of search list.
a.c:1:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
main(){}
^~~~
1 warning generated.
"/usr/bin/ld" --eh-frame-hdr -m elf_i386 -dynamic-linker /lib/ld-linux.so.2 -o a.out crt1.o crti.o crtbegin.o -L -L/../../.. /tmp/cc-2ueoYy.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed crtend.o crtn.o
/usr/bin/ld: crt1.o: No such file: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The same error was here http://permalink.gmane.org/gmane.comp.compilers.clang.devel/12101
and http://comments.gmane.org/gmane.comp.compilers.clang.devel/14269 and http://permalink.gmane.org/gmane.comp.compilers.llvm.bugs/11352
What I can learn while comparing with llvm-2.8 -- newer version tries to assemble file itself, and older version calls gcc to do assemble and link steps.
Please, this is not a duplicate of clang linker problem as I can't edit sources of LLVM, so the accepted solution isn't useful to me.
As suggested at http://permalink.gmane.org/gmane.comp.compilers.llvm.bugs/11352 (llvm bug 8897) this is because
gcc used to build clang is not in a standard location
or
system gcc is not listed in the ToolChains.cpp GccVersions list:
./tools/clang/lib/Driver/ToolChains.cpp: const char* GccVersions[] = {"4.5.2", "4.5.1", "4.5", "4.4.5", "4.4.4",
./tools/clang/lib/Driver/ToolChains.cpp- "4.4.3", "4.4", "4.3.4", "4.3.3", "4.3.2",
./tools/clang/lib/Driver/ToolChains.cpp- "4.3", "4.2.4", "4.2.3", "4.2.2", "4.2.1",
./tools/clang/lib/Driver/ToolChains.cpp- "4.2"};
Both cases are true for my installation.
To fix this, I'll try to add my version of gcc into list GccVersions[].
This helps me.