linker (ld) on OS X: How to use -Wl,--start-group (and --end-group)? - c++
I am building some projects externally which use libclang static libs.
Linking fails like this:
ld: unknown option: --start-group
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The reason I need to use this is because of circular dependencies (or as it turns out, I guess because of not knowing the correct library order).
So far I have to resort to taking the -Wl,--{start,end}-group out of the makefile, looking at the undefined symbols error list, finding the libs that have them with nm, and appending them manually to the library list (so they appear more than once in the library list).
The correct order of LLVM/Clang libs (taken from my own project that uses it) for 3.5 is:
"clangFrontend",
"clangSerialization",
"clangDriver",
"clangTooling",
"clangCodeGen",
"clangParse",
"clangSema",
"clangAnalysis",
"clangRewriteFrontend",
"clangRewrite",
"clangEdit",
"clangAST",
"clangLex",
"clangBasic",
"LLVMLTO",
"LLVMObjCARCOpts",
"LLVMLinker",
"LLVMipo",
"LLVMVectorize",
"LLVMBitWriter",
"LLVMIRReader",
"LLVMAsmParser",
"LLVMR600CodeGen",
"LLVMR600Desc",
"LLVMR600Info",
"LLVMR600AsmPrinter",
"LLVMSystemZDisassembler",
"LLVMSystemZCodeGen",
"LLVMSystemZAsmParser",
"LLVMSystemZDesc",
"LLVMSystemZInfo",
"LLVMSystemZAsmPrinter",
"LLVMHexagonCodeGen",
"LLVMHexagonAsmPrinter",
"LLVMHexagonDesc",
"LLVMHexagonInfo",
"LLVMNVPTXCodeGen",
"LLVMNVPTXDesc",
"LLVMNVPTXInfo",
"LLVMNVPTXAsmPrinter",
"LLVMCppBackendCodeGen",
"LLVMCppBackendInfo",
"LLVMMSP430CodeGen",
"LLVMMSP430Desc",
"LLVMMSP430Info",
"LLVMMSP430AsmPrinter",
"LLVMXCoreDisassembler",
"LLVMXCoreCodeGen",
"LLVMXCoreDesc",
"LLVMXCoreInfo",
"LLVMXCoreAsmPrinter",
"LLVMMipsDisassembler",
"LLVMMipsCodeGen",
"LLVMMipsAsmParser",
"LLVMMipsDesc",
"LLVMMipsInfo",
"LLVMMipsAsmPrinter",
"LLVMAArch64Disassembler",
"LLVMAArch64CodeGen",
"LLVMAArch64AsmParser",
"LLVMAArch64Desc",
"LLVMAArch64Info",
"LLVMAArch64AsmPrinter",
"LLVMAArch64Utils",
"LLVMARMDisassembler",
"LLVMARMCodeGen",
"LLVMARMAsmParser",
"LLVMARMDesc",
"LLVMARMInfo",
"LLVMARMAsmPrinter",
"LLVMPowerPCDisassembler",
"LLVMPowerPCCodeGen",
"LLVMPowerPCAsmParser",
"LLVMPowerPCDesc",
"LLVMPowerPCInfo",
"LLVMPowerPCAsmPrinter",
"LLVMSparcDisassembler",
"LLVMSparcCodeGen",
"LLVMSparcAsmParser",
"LLVMSparcDesc",
"LLVMSparcInfo",
"LLVMSparcAsmPrinter",
"LLVMTableGen",
"LLVMDebugInfo",
"LLVMOption",
"LLVMX86Disassembler",
"LLVMX86AsmParser",
"LLVMX86CodeGen",
"LLVMSelectionDAG",
"LLVMAsmPrinter",
"LLVMX86Desc",
"LLVMX86Info",
"LLVMX86AsmPrinter",
"LLVMX86Utils",
"LLVMJIT",
"LLVMLineEditor",
"LLVMMCAnalysis",
"LLVMMCDisassembler",
"LLVMInstrumentation",
"LLVMInterpreter",
"LLVMCodeGen",
"LLVMScalarOpts",
"LLVMInstCombine",
"LLVMTransformUtils",
"LLVMipa",
"LLVMAnalysis",
"LLVMProfileData",
"LLVMMCJIT",
"LLVMTarget",
"LLVMRuntimeDyld",
"LLVMObject",
"LLVMMCParser",
"LLVMBitReader",
"LLVMExecutionEngine",
"LLVMMC",
"LLVMCore",
"LLVMSupport"
You don't have to determine the order yourself- use -llvm-config to get the LLVM order. The Clang order is a bit trickier- from memory, you need to extract it from makefiles used to build Clang itself, or something like that. However, the Clang list is pretty small so determining it is pretty easy given the LLVM order and that Clang must go before LLVM. I don't know where libclang goes in this list since I don't use it, but I'm guessing that it should go first.
The accepted answer by #Puppy does answer the OP's ultimate need -- how to get the libraries in required link order.
But it does not answer the OP's actual question -- how to use --start-group and --end-group.
The clang and gcc command line options are:
-Wl,--start-group
libs-in-the-order-you-need ...
-Wl,--end-group
Your mileage may vary on other compilers.
UPDATE (2019.07.11):
David Given points out that the gcc provided by default on OSX doesn't even support these options.
That is because OSX only provides gcc 2.x compatibility.
I was using gcc 4.8 and gcc 7.0 when I started using these options in my project.
I am not sure when they were first added to gcc.
Related
What are mechanisms of selection of emulation mode for a linker?
I am trying to compile arduino code under codeblocks with avr-gnu compiler. I was getting this error: avr-ld.exe unrecognized emulation mode: mcu=atmega328p. Supported emulations:avr2 avr1 avr25 avr3 avr31 avr35 avr4 avr5 avr51 avr6 avrxmega1 avrxmega2 avrxmega3 avrxmega4 avrxmega5 avrxmega6 avrxmega7 avrtiny I found this page: https://gcc.gnu.org/onlinedocs/gcc/AVR-Options.html which suggests that it should be linked under avr5 sim mode, so I added -mmcu=avr5 to linker options. Now I am getting: avr-ld.exe unrecognized emulation mode: mcu=avr5 Supported emulations:avr2 avr1 avr25 avr3 avr31 avr35 avr4 avr5 avr51 avr6 avrxmega1 avrxmega2 avrxmega3 avrxmega4 avrxmega5 avrxmega6 avrxmega7 avrtiny How is avr5 unrecognized if it is supported?o0 I tried few different toolchains from few different years but it seems that linker cant interpret this emulation mode correctly. Newer toolchains had support for 328p but still somehow dont work. I also tried creating CodeBlocks project dedicated for Arduino but the difference is that it passes -mmcu=atmega328p itself and still doesnt work on this or newer toolchains. Update: There is something that I am totally missing here. I changed linker to a standard x86 32 ld.exe from Mingw and passed -mmcu=i386pe and I am getting the similar error: ld.exe unrecognized emulation mode: mcu=i386pe Supported emulations:i386pe So this only leads to my lack of understanding of mechanisms how the linker emulation mode is selected/configured.
In almost any case, you do not want to call the linker (ld resp. avr-ld) by hand!
Use avr-gcc to link your programs!
Notice that avr-gcc is not a compiler by itself, it's just a driver program that calls sub-programs like compiler proper (cc1 for C, cc1plus for C++, lto1 for LTO-compiled byte-code), assembler and linker depending on command-line options and input files provided on the command line.
avr-gcc knows how to provide the respective tools with appropriate command line options like: emulation, libraries, paths for includes and libraries, startup-code, sub-options, etc.
In order to see which tools are called by avr-gcc, add the -v option. For example, for a compile command as simple as
> avr-gcc -v main.c -mmcu=atmega328p -save-temps
You'll see somethinh like (here for avr-gcc v8.5.0):
...
COLLECT_GCC_OPTIONS='-v' '-save-temps' '-specs=device-specs/specs-atmega328p' '-mmcu=avr5'
$AVRGCC_HOME/bin/../libexec/gcc/avr/8.5.0/collect2 <many options> -plugin-opt=-pass-through=-latmega328p -mavr5 -Tdata 0x800100 $AVRGCC_HOME/bin/../lib/gcc/avr/8.5.0/../../../../avr/lib/avr5/crtatmega328p.o -L$AVRGCC_HOME/bin/../lib/gcc/avr/8.5.0/avr5 -L$AVRGCC_HOME/bin/../lib/gcc/avr/8.5.0/../../../../avr/lib/avr5 -L$AVRGCC_HOME/bin/../lib/gcc/avr/8.5.0 -L$AVRGCC_HOME/bin/../lib/gcc -L$AVRGCC_HOME/bin/../lib/gcc/avr/8.5.0/../../../../avr/lib main.o --start-group -lgcc -lm -lc -latmega328p --end-group
collect2 is just some other tool that wraps the call to ld. If you want to see the real thing, use avr-gcc ... -v -Wl,-v.
The correct emulation for the linker is worked out by the device-specific spec file device-specs/specs-atmega328p located at ./lib/gcc/avr/$version in the compiler's install directory. The specs "language" is a means to describe option mappings; in particular the bits that are responsible for mapping `-mmcu=atmega328p' to the right emulation are
*link_arch:
%{mmcu=*:-m%*}
...
*self_spec:
%{!mmcu=avr*: %<mmcu=* -mmcu=avr5} %<mshort-calls %<msp8
self_spec removes -mmcu=* from the command line and sets -mmcu=avr5 which is the core family ATmega328P belongs to. Then as part of the linker invocation, link_arch maps -mmcu=avr5 to -mavr5 which is the linker emulation to be used.
Cray compiler asks for a flag, then doesn't recognize it
I am trying to compile a code with crayftn. I get an error message /opt/cray/pe/cce/10.0.1/binutils/x86_64/x86_64-pc-linux-gnu/bin/ld: failed to convert GOTPCREL relocation; relink with --no-relax So it wants the flag --no-relax? OK, I can do that. So I re-link with that flag, and then it tells me ftn -O3 --no-relax -dynamic -h pic -h omp -o stream_cray stream_mpi.o mysecond.o ftn-2115 crayftn: ERROR in command line "-no-relax" is an invalid command-line option. So it asks for "--no-relax", but then it doesn't understand it. Anyone know of a way out of this conundrum? Or another way of solving the root problem in the first place?
I found this link: https://bb.cgd.ucar.edu/cesm/threads/failed-to-convert-gotpcrel-relocation-relink-with-no-relax.4494/ PROBLEM: Hi, I get the following error message for CLM5.0 compilation with Intel compilers, during the final cesm bld ..ld: failed to convert GOTPCREL relocation; relink with --no-relax SOLUTION: Hi, Seems like found a solution to fix the compilation ...Adding of "-Wl,--no-relax" in LDFLAGS does not solve this problem, but "-mcmodel medium" in FFLAGS fixes this issue, after searching for "Relocation truncated to fit" in google search engine, it comes up with this link which was helpful to solve the issue "https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x/topic/268394" Best Regards,Prabhakar See also: https://community.intel.com/t5/Intel-Fortran-Compiler/relocation-truncated-to-fit/td-p/1146616 This looks like mixing compilers and libraries from different systems mixed up, either 32bit vs. 64bit or installations for ifort and mpiifort based on different glibc or something similar. I'm still curious about your "development environment": Q: Have you been able to successfully compile, link and run ANY program with your crayftn? Q: What version of crayftn? 10.0.1? Q: What platform? Where is x86_64-pc-linux coming from? Just curious...
Source for 'startup files' in gcc/mingw
Im using mingw32 In gcc there is an option " -nostartfiles Do not use the standard system startup files when linking. The standard system libraries are used normally, unless -nostdlib or -nodefaultlibs is used. " If i do not use it in my itself almost empty c source i got in my exe 1000-lines disasembly long stub of code containing such functions 00401000 <___mingw_CRTStartup>: 00401110 <__gnu_exception_handler#4>: 00401280 <_mainCRTStartup>: 004012a0 <_WinMainCRTStartup>: 004012c0 <_atexit>: 004012d0 <__onexit>: 004012e0 <___gcc_register_frame>: 0040132c <___gcc_deregister_frame>: 00401334 <_WinMain#16>: 00401340 <___dyn_tls_dtor#12>: 00401390 <___dyn_tls_init#12>: 00401420 <___tlregdtor>: 00401430 <___cpu_features_init>: 00401510 <__fpreset>: 00401520 <___report_error>: 00401580 <___write_memory.part.0>: 00401690 <__pei386_runtime_relocator>: 00401870 <___do_global_dtors>: 004018a0 <___do_global_ctors>: 004018f0 <___main>: 00401910 <___mingwthr_run_key_dtors.part.0>: 00401970 <____w64_mingwthr_add_key_dtor>: 00401a00 <____w64_mingwthr_remove_key_dtor>: 00401a90 <___mingw_TLScallback>: 00401b30 <___getmainargs>: 00401b38 <__setmode>: 00401b40 <___p__fmode>: 00401b48 <___p__environ>: 00401b50 <__cexit>: 00401b58 <_signal>: 00401b60 <_fwrite>: 00401b68 <_vfprintf>: 00401b70 <_abort>: 00401b78 <_memcpy>: 00401b80 <_calloc>: 00401b88 <_free>: 00401b90 <_SetUnhandledExceptionFilter#4>: 00401b98 <_ExitProcess#4>: 00401ba0 <_GetModuleHandleA#4>: 00401ba8 <_GetProcAddress#8>: 00401bb0 <_VirtualQuery#12>: 00401bb8 <_VirtualProtect#16>: 00401bc0 <_GetCommandLineA#0>: 00401bc8 <_GetStartupInfoA#4>: 00401bd0 <_EnterCriticalSection#4>: 00401bd8 <_TlsGetValue#4>: 00401be0 <_GetLastError#0>: 00401be8 <_LeaveCriticalSection#4>: 00401bf0 <_DeleteCriticalSection#4>: 00401bf8 <_InitializeCriticalSection#4>: 00401c00 <_main>: 00401ce0 <_register_frame_ctor>: 00401cfc <__CTOR_LIST__>: 00401d00 <.ctors.65535>: 00401d08 <__DTOR_LIST__>: Could maybe someone say where i could find source for that? (or at least some detailed explanation what is that - but source would be better)
I guess you could download GCC sources or MinGW sources. Since the mingw-w64 project on sourceforge.net is moving to mingw-w64.org I suggest also to check mingw-w64.org and then run grep or similar text search utility and find particular file(s) where those function(s) are defined. Note also that some functions like exit(), atexit(), memcpy(), etc. are coming from C Run-Time Library, they won't be defined in GCC sources.
Compile c++ code in R does not work anymore
I start saying that i am a newbie in programming and then i am not sure i will be able to explain well my problem.
I had some c++ code i wrote, this code are loaded and used by some R functions.
To compile the code i used the following:
R CMD SHLIB MyCode.cpp
and i loaded the library in R with
dyn.load("MyCOde.so")
Sometimes i built also an R package and i was able to load it into R.
If i do all these stuff on a Mac with mountain lion everything work fine, but now that i switched to mavericks, i have some problems. The R CMD SHLIB MyCode.cpp command works but when i used dyn.load("MyCOde.so") i get the following text:
Errore in dyn.load(paste(dir_function, "MyCOde.so", sep = "")) :
unable to load shared object 'MyCOde.so':
dlopen(MyCOde.so, 6): Symbol not found: __ZNSt8ios_base4InitC1Ev
Referenced from: MyCOde.so
Expected in: flat namespace
in MyCOde.so
Moreover if i try to load the package in R, i get the following
ld: warning: directory not found for option '-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2'
ld: library not found for -lquadmath
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [MyCode.so] Error 1
Can someone helps me?
Based on the helpful website of: thecoatlessprofessor Type this into your terminal shell: curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2 sudo tar fvxz gfortran-4.8.2-darwin13.tar.bz2 -C / This will create what you need to resume compiling as before.
Since it starts to work I can publish the answer for such a cases. When you change the compiler and standard libraries - please note that different libraries have different implementation and different standard support. Changing the basement of your system might require total rebuild of your system with the new C++ standard library. Your libraries are not the exception. So if have the errors in your linker like this: warning: directory not found for option '-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2' apply next algorithm: Check whether the directory /usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2 still exists. I bet it is not. Check if you still have the libstdc++ from the missed compiler? Usually if you upgrade the same compiler and the C++ standard library ABI does not change everything should continue to work. If the ABI changed or you switch standard C++ library and compiler - you face the massive system rebuild. Recompile your library and apps with the new C++ standard library and compiler.
LLVM libc++ not compiling with clang 3.3 on Mac OS
I have just downloaded clang 3.3 (homebrew) from the LLVM web page to my mac (OS X 10.8.4), but get this compiler error when using std=c++11 stdlib=libc++: In file included from /usr/include/c++/v1/string:434: In file included from /usr/include/c++/v1/algorithm:594: In file included from /usr/include/c++/v1/memory:590: In file included from /usr/include/c++/v1/typeinfo:61: /usr/include/c++/v1/exception:146:5: error: an attribute list cannot appear here _LIBCPP_NORETURN friend void rethrow_exception(exception_ptr); ^~~~~~~~~~~~~~~~ /usr/include/c++/v1/__config:190:28: note: expanded from macro '_LIBCPP_NORETURN' # define _LIBCPP_NORETURN [[noreturn]] ^~~~~~~~~~~~ It seems that I also need another libc++ (even though it was said that it was 100% complete on MAC ...), but I cannot find any. Any help appreciated. Just for your info: > clang++ -v clang version 3.3 (tags/RELEASE_33/final) Target: x86_64-apple-darwin12.4.0 Thread model: posix And, yes, I googled it and found this: http://comments.gmane.org/gmane.comp.compilers.llvm.bugs/24138 claiming it's resolved in libc++ trunk ??? Okay, as suggested by Howard, I've downloaded tip-of-the-trunk libc++ into /opt/local/share/libcxx, but have trouble building it. The manual says to cd libcxx/lib, export TRIPLE=-apple-, and run ./buildit. I presume this implies bash (I'm usually a tcsh user, so I moved my .tcshrc, got a new shell and started bash). I did that and the compilations worked, but the library build failed. Apparently ./buildit doesn't see $TRIPLE=-apple-, as it picks the wrong LDSHARED_FLAG (not that on line 81, but that on line 103, which is to be used if $TRIPLE is not set), even though echo $TRIPLE yields -apple- as it should. When I add the statement echo TRIPLE = $TRIPLE at the top of buildit, it reports nothing. How come? What is wrong here? The failure was that because the wrong LDSHARED_FLAG was picked the loading didn't work (ld complaint about the unknown option -soname which, I think, makes sense under linux). I don't know why buildit (a #! /bin/sh file) didn't pick up the TRIPLE environment variable (it did pick up several unwanted ones such as CXX and CC). I now simply added TRIPLE=-apple- at the top of that file and it did built the library. However, the loader spitted out several warnings all of which were of the form ld: warning: direct access in ___cxa_bad_typeid to global weak symbol typeinfo for std::bad_typeid means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings. But most importantly, it works (the compilation at least, I have yet to test the library). I have one final question. The advice was to use -I and -L to tell the compiler about the whereabouts of this version. Is it not possible to put it into the usual place /usr/include/c++/v1/? Note that Xcode has its version somewhere else anyway and I had put in a symbolic link (/usr/include/c++/v1/) to that one to get my homebrew clang 3.2 working (after the some Xcode update). What about the library? Can I also put it in a standard place?
Here is the home page of libc++: http://libcxx.llvm.org You can download the tip-of-trunk libc++ from there. You can tell clang to point to your download with -nostdinc++ -I<path-to-libc++>/include. You can also tell clang to link to your tip-of-trunk libc++ with -L<path-to-libc++>/lib and export DYLD_LIBRARY_PATH=<path-to-libcxx>/lib. The directions are all on the libc++ home page. Xcode is the easiest way to get clang + libc++. But if you want the very latest, this is the place to go. Congratulations! Don't worry about the ld warning. It is a harmless ld bug that will be fixed in a future release. I see it on 10.8.4 too and it doesn't hurt anything. The libc++ headers no longer live at /usr/include/c++/v1. Xcode has migrated them into itself. Having libc++ headers at /usr/include/c++/v1 from older installs has been a source of confusion and bugs. I regularly use -nostdinc++ -I to point to the libc++ headers I want (I often have several versions going at the same time), and that works well for me. It is possible for you to replace your /usr/lib/libc++.1.dylib with that you have built. I do not recommend doing this. I have to sometimes to do a proper test, but I always do so very carefully because sometimes this causes me to have to reboot onto a backup disk and restore my /usr/lib to its original state. If you do go this route, it is a very good idea to have a backup of the original /usr/lib/libc++.1.dylib very handy. I recommend instead -L on the command line, and export DYLD_LIBRARY_PATH=<path-to-libcxx>/lib in the shell. More than one person (including myself) has gotten their computer into a really nasty place by not following this advice. If you run testit (under test/), all you need is DYLD_LIBRARY_PATH in that shell. The testit script is set up to point to the right places without an install. Also I recommend figuring out why you had to modify buildit. No one else is seeing that behavior. printenv on your command line may help in this endeavor. libc++ is updated often. We try to keep tip-of-trunk always in a shippable state.