llc: unsupported relocation on symbol - llvm

Problem
llc is giving me the following error:
LLVM ERROR: unsupported relocation on symbol
Detailed compilation flow
I am implementing an LLVM frontend for a middle-level IR (MIR) of a compiler, and after I convert various methods to many bitcode files, I link them (llvm-link), optimize them (opt), convert them to machine code (llc), make them a shared library (clang for it's linker wrapper), and dynamically load them.
llc step fails for some of the methods that I am compiling!
Step 1: llvm-link: Merge many bitcode files
I may have many functions calling each other, so I llvm-link the different bitcode files that might interact with each other. This step has no issues. Example:
llvm-link function1.bc function2.bc -o lnk.bc
Step 2: opt: Run optimization passes
For now I am using the following:
opt -O3 lnk.bc -o opt.bc
This step proceeds with no issues, but that's the one that CAUSES the problem!
Also, it's necessary because in the future I will need this step to pass extra passes, e.g. loop-unroll
Step 3: llc: Generate machine code (PIC)
I am using the following command:
llc -march=thumb -arm-reserve-r9 -mcpu=cortex-a9 -filetype=obj -relocation-model pic opt.bc -o obj.o
I have kept the arch specific flags I've set just in case they contribute to the issue. I am using Position Independent Code because on next step I will be building a shared object.
This command fails with the error I've written on top of this answer.
Step 4: clang: Generate Shared Object
For the cases that Step 3 fails, this step isn't reached.
If llc succeeds, this step will succeed too!
Additional information
Configuration
The following run on an llvm3.6, which runs on an arm device.
Things I've noticed
If I omit -O3 (or any other level) with the opt step, then llc would work.
If I don't, and instead I omit them from llc, llc would still fail. Which makes me think that opt -O<level> is causing the issue.
If I use llc directly it will work, but I won't be able to run specific passes that opt allows me, so this is not a option for me.
I've faced this issue ONLY with 2 functions that I've compiled so far (from their original MIR), which use loops. The others produce working code!
If I don't use pic model at llc, it can generate obj.o, but then I'll have problems creating an .so from it!
Questions
Why is this happening??!!
Why opt has -relocation-model option? Isn't that supposed to be just an llc thing? I've tried setting this both at opt and llc to pic, but still fails.
I am using clang because it has a wrapper to a linker to get the the .so. Is there a way to do this step with an LLVM tool instead?

First of all, do not use neither llc nor opt. These are developer-side tools that should be never used in any production environment. Instead of this, implement your own proper optimization and codegeneration runtime via LLVM libraries.
As for this particular bug - the thumb codegenerator might contain some bugs. Please reduce the problem and report it. Or don't use Thumb mode at all :)

Related

Cray compiler asks for a flag, then doesn't recognize it

I am trying to compile a code with crayftn.
I get an error message
/opt/cray/pe/cce/10.0.1/binutils/x86_64/x86_64-pc-linux-gnu/bin/ld: failed to convert GOTPCREL relocation; relink with --no-relax
So it wants the flag --no-relax? OK, I can do that. So I re-link with that flag, and then it tells me
ftn -O3 --no-relax -dynamic -h pic -h omp -o stream_cray stream_mpi.o mysecond.o
ftn-2115 crayftn: ERROR in command line
"-no-relax" is an invalid command-line option.
So it asks for "--no-relax", but then it doesn't understand it. Anyone know of a way out of this conundrum? Or another way of solving the root problem in the first place?
I found this link:
https://bb.cgd.ucar.edu/cesm/threads/failed-to-convert-gotpcrel-relocation-relink-with-no-relax.4494/
PROBLEM:
Hi, I get the following error message for CLM5.0 compilation with
Intel compilers, during the final cesm bld ..ld: failed to convert GOTPCREL relocation; relink with --no-relax
SOLUTION:
Hi, Seems like found a solution to fix the compilation ...Adding of
"-Wl,--no-relax" in LDFLAGS does not solve this problem, but
"-mcmodel medium" in FFLAGS fixes this issue, after searching for
"Relocation truncated to fit" in google search engine, it comes up
with this link which was helpful to solve the issue
"https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x/topic/268394"
Best Regards,Prabhakar
See also:
https://community.intel.com/t5/Intel-Fortran-Compiler/relocation-truncated-to-fit/td-p/1146616
This looks like mixing compilers and libraries from different systems
mixed up, either 32bit vs. 64bit or installations for ifort and
mpiifort based on different glibc or something similar.
I'm still curious about your "development environment":
Q: Have you been able to successfully compile, link and run ANY
program with your crayftn? Q: What version of crayftn? 10.0.1? Q: What
platform? Where is x86_64-pc-linux coming from? Just curious...

How to use the -pgo-instr-gen arguments with llvm opt

I want to test the PGO with llvm and I find most of the tutorial in the internet uses clang with argument -fprofile-instr-use to enable PGO instrumentation. However, what I have is an llvm bitcode file instead of the source code and I want to apply PGO to these bitcode files. I noticed the opt tool has the argument -pgo-instr-gen.
However, after I applied it to my bitcode like:
opt -pgo-instr-gen input.ll -o output.ll
and then tried to run the output.ll with lli, I got a segmentation fault. So, what is the correct way to enable PGO in such cases?

Ocamlopt doesn’t produce any output, only an error code

I’m trying to call into a bulky C++ library from OCaml, and I’m having trouble with ocamlopt, which sitently fails with error code 2.
I’m doing the whole dance with putting up a C interface, and I can get it to work in general, but as soon as I reference this library, build breaks.
Is there some way to know what exactly is failing? I tried the -verbose flag, but it just prints the commandline arguments (which are quite long).
Would you have any tips as to how to investigate a silent failure like this?
TL;DR; check that you have enough memory and/or disk space.
Something like this could happen when ocamlopt is either killed by a signal or runs out the memory (or both), check the dmesg output, look for OOM messages from the kernel, also use htop to get the idea on the memory footprint.
Also, since this happens when you're trying to link with the C++ library, it is most likey that it is the ld process which is failing (again, most likely with OOM), as ocamlopt uses the system linker.
In case anyone else runs into this again: the problem was that there were too many -ccopt and -cclib arguments getting passed in by the build driver. When I started including a C++ library with lots of other dependencies, we seemed to have reached the breaking point.
The solution was to change the build driver's OCaml compiler and linker rules to write all the compiler and linker args into files so they could all be passed in as a single -ccopt #<compiler.args> or -cclib #<linker.args> argument. Both gcc and ld support the #file command line option.
GitHub issue: ocamlopt lets the compiler/linker silently fail if too many -ccopt or -cclib arguments are passed in

linker (ld) on OS X: How to use -Wl,--start-group (and --end-group)?

I am building some projects externally which use libclang static libs.
Linking fails like this:
ld: unknown option: --start-group
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The reason I need to use this is because of circular dependencies (or as it turns out, I guess because of not knowing the correct library order).
So far I have to resort to taking the -Wl,--{start,end}-group out of the makefile, looking at the undefined symbols error list, finding the libs that have them with nm, and appending them manually to the library list (so they appear more than once in the library list).
The correct order of LLVM/Clang libs (taken from my own project that uses it) for 3.5 is:
"clangFrontend",
"clangSerialization",
"clangDriver",
"clangTooling",
"clangCodeGen",
"clangParse",
"clangSema",
"clangAnalysis",
"clangRewriteFrontend",
"clangRewrite",
"clangEdit",
"clangAST",
"clangLex",
"clangBasic",
"LLVMLTO",
"LLVMObjCARCOpts",
"LLVMLinker",
"LLVMipo",
"LLVMVectorize",
"LLVMBitWriter",
"LLVMIRReader",
"LLVMAsmParser",
"LLVMR600CodeGen",
"LLVMR600Desc",
"LLVMR600Info",
"LLVMR600AsmPrinter",
"LLVMSystemZDisassembler",
"LLVMSystemZCodeGen",
"LLVMSystemZAsmParser",
"LLVMSystemZDesc",
"LLVMSystemZInfo",
"LLVMSystemZAsmPrinter",
"LLVMHexagonCodeGen",
"LLVMHexagonAsmPrinter",
"LLVMHexagonDesc",
"LLVMHexagonInfo",
"LLVMNVPTXCodeGen",
"LLVMNVPTXDesc",
"LLVMNVPTXInfo",
"LLVMNVPTXAsmPrinter",
"LLVMCppBackendCodeGen",
"LLVMCppBackendInfo",
"LLVMMSP430CodeGen",
"LLVMMSP430Desc",
"LLVMMSP430Info",
"LLVMMSP430AsmPrinter",
"LLVMXCoreDisassembler",
"LLVMXCoreCodeGen",
"LLVMXCoreDesc",
"LLVMXCoreInfo",
"LLVMXCoreAsmPrinter",
"LLVMMipsDisassembler",
"LLVMMipsCodeGen",
"LLVMMipsAsmParser",
"LLVMMipsDesc",
"LLVMMipsInfo",
"LLVMMipsAsmPrinter",
"LLVMAArch64Disassembler",
"LLVMAArch64CodeGen",
"LLVMAArch64AsmParser",
"LLVMAArch64Desc",
"LLVMAArch64Info",
"LLVMAArch64AsmPrinter",
"LLVMAArch64Utils",
"LLVMARMDisassembler",
"LLVMARMCodeGen",
"LLVMARMAsmParser",
"LLVMARMDesc",
"LLVMARMInfo",
"LLVMARMAsmPrinter",
"LLVMPowerPCDisassembler",
"LLVMPowerPCCodeGen",
"LLVMPowerPCAsmParser",
"LLVMPowerPCDesc",
"LLVMPowerPCInfo",
"LLVMPowerPCAsmPrinter",
"LLVMSparcDisassembler",
"LLVMSparcCodeGen",
"LLVMSparcAsmParser",
"LLVMSparcDesc",
"LLVMSparcInfo",
"LLVMSparcAsmPrinter",
"LLVMTableGen",
"LLVMDebugInfo",
"LLVMOption",
"LLVMX86Disassembler",
"LLVMX86AsmParser",
"LLVMX86CodeGen",
"LLVMSelectionDAG",
"LLVMAsmPrinter",
"LLVMX86Desc",
"LLVMX86Info",
"LLVMX86AsmPrinter",
"LLVMX86Utils",
"LLVMJIT",
"LLVMLineEditor",
"LLVMMCAnalysis",
"LLVMMCDisassembler",
"LLVMInstrumentation",
"LLVMInterpreter",
"LLVMCodeGen",
"LLVMScalarOpts",
"LLVMInstCombine",
"LLVMTransformUtils",
"LLVMipa",
"LLVMAnalysis",
"LLVMProfileData",
"LLVMMCJIT",
"LLVMTarget",
"LLVMRuntimeDyld",
"LLVMObject",
"LLVMMCParser",
"LLVMBitReader",
"LLVMExecutionEngine",
"LLVMMC",
"LLVMCore",
"LLVMSupport"
You don't have to determine the order yourself- use -llvm-config to get the LLVM order. The Clang order is a bit trickier- from memory, you need to extract it from makefiles used to build Clang itself, or something like that. However, the Clang list is pretty small so determining it is pretty easy given the LLVM order and that Clang must go before LLVM. I don't know where libclang goes in this list since I don't use it, but I'm guessing that it should go first.
The accepted answer by #Puppy does answer the OP's ultimate need -- how to get the libraries in required link order.
But it does not answer the OP's actual question -- how to use --start-group and --end-group.
The clang and gcc command line options are:
-Wl,--start-group
libs-in-the-order-you-need ...
-Wl,--end-group
Your mileage may vary on other compilers.
UPDATE (2019.07.11):
David Given points out that the gcc provided by default on OSX doesn't even support these options.
That is because OSX only provides gcc 2.x compatibility.
I was using gcc 4.8 and gcc 7.0 when I started using these options in my project.
I am not sure when they were first added to gcc.

How to make the Clang Static Analyzer output its working from command line?

I'm running Clang 3.4 on Ubuntu 12.10 (from http://llvm.org/apt/). I ran the analyzer (clang --analyze) over some code, and it found a couple of issues:
Blah.C:429:9: warning: Declared variable-length array (VLA) has zero size
unsigned char separatedData[groupDataLength];
^~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
But the specific issue isn't important. I want to know the steps of how it came to that conclusion (the code is complex enough for me not to see it within 15 mins).
I see a screenshot from the Clang site that shows steps of working viewed in a web browser:
That's probably obtained from Xcode.
The question is: how do I get Clang to output such steps of working from the command line? Or even output results to a browser if it so wishes? This would make the analyzer significantly more useful, and make fixing things much quicker.
(I have noticed that GCC's documentation is very excellent, but Clang/LLVM's documentation is very poor. I've tried "clang --analyze -Xanalyzer '-v'" as a stab in the dark to tell the analyzer to be more verbose -- the -Xanalyzer switch was from the man pages.)
In addition to text output on the console:
clang++ --analyze -Xanalyzer -analyzer-output=text main.cpp
You can get the full html output:
clang++ --analyze -Xanalyzer -analyzer-output=html -o html-dir main.cpp
Additionally, you can select specific checkers to enable. This page lists available checks. For example, you can enable all of the C++ checks in the alpha group using the flags:
-Xanalyzer -analyzer-checker=alpha.cplusplus
http://coliru.stacked-crooked.com/a/7746c4004704d4a7
main.cpp:5:1: warning: Potential leak of memory pointed to by 'x'
}
^
main.cpp:4:12: note: Memory is allocated
int *x = new int;
^~~~~~~
main.cpp:5:1: note: Potential leak of memory pointed to by 'x'
}
^
Apparently the front end exposes
-analyzer-config <Option Name>=<Value>
E.g.
-analyzer-config -analyzer-checker=alpha.cplusplus
which might be better supported than -Xanalyzer and may be getting extended to support options to individual checkers: http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-October/039552.html
You are on the right track, but to get the full trace leading to a bug you additionally need to ask clang for output in text format (don't ask why). Since you will probably need to adjust e.g. include paths or defines for your project anyway I'd suggest you use clang-check which acts as a wrapper around clang's analyzer pass. It can also hook into the static analyzer tools exposed in e.g. scan-build. You can then
$ clang-check -analyze -extra-arg -Xclang -extra-arg -analyzer-output=text
Like you wrote the documentation for these very nice tools is abysmal. I cobbled above call together from bits and pieces from Chandler Carruth's GoingNative2013 talk.
You have to use scanbuild: http://clang-analyzer.llvm.org/scan-build.html
You type the commands that generate your build, but you pre-pend them with scan-build.
Example:
instead of
make
type
scan-build make
instead of
./configure
make
type
scan-build ./configure
scan-build make
Clear the build before launching the analyzer, otherwise make will state that everything has been built already and the analyzer will not run.