I was told that clang is a driver that works like gcc to do preprocessing, compilation and linkage work. During the compilation and linkage, as far as I know, it's actually llvm that does the optimization ("-O1", "-O2", "-O3", "-Os", "-flto").
But I just cannot understand how llvm is involved.
It seems that compiling source code doesn't even need a static library such as libLLVMCore.a, instead for debian clang packages depends on another package called libllvm-3.4(clang version is 3.4), which contains libLLVM-3.4.so(.1), does clang use this shared library for optimization?
I've checked clang source code for a while and found that include/clang/Driver/Options.td contains the related options, but unfortunately I failed to find the source files that include that file, so I'm still not aware of the mechanism.
I hope someone might give me some hints.
(TL;DontWannaRead - skip to the end of this answer)
To answer your question properly you first need to understand the difference between a compiler's front-end and back-end (especially the first one).
Clang is a compiler front-end (http://en.wikipedia.org/wiki/Clang) for C, C++, Objective C and Objective C++ languages.
Clang's duty is the following:
i.e. translating from C++ source code (or C, or Objective C, etc..) to LLVM IR, a textual lower-level representation of what should that code do. In order to do this Clang employs a number of sub-modules whose descriptions you could find in any decent compiler construction book: lexer, parser + a semantic analyzer (Sema), etc..
LLVM is a set of libraries whose primary task is the following: suppose we have the LLVM IR representation of the following C++ function
int double_this_number(int num) {
int result = 0;
result = num;
result = result * 2;
return result;
}
the core of the LLVM passes should optimize LLVM IR code:
What to do with the optimized LLVM IR code is entirely up to you: you can translate it to x86_64 executable code or modify it and then spit it out as ARM executable code or GPU executable code. It depends on the goal of your project.
The term "back-end" is often confusing since there are many papers that would define the LLVM libraries a "middle end" in a compiler chain and define the "back end" as the final module which does the code generation (LLVM IR to executable code or something else which no longer needs processing by the compiler). Other sources refer to LLVM as a back end to Clang. Either way, their role is clear and they offer a powerful mechanism: whatever the language you're targeting (C++, C, Objective C, Python, etc..) if you have a front-end which translates it to LLVM IR, you can use the same set of LLVM libraries to optimize it and, as long as you have a back-end for your target architecture, you can generate optimized executable code.
Recalling that LLVM is a set of libraries (not just optimization passes but also data structures, utility modules, diagnostic modules, etc..), Clang also leverages many LLVM libraries during its front-ending process. You can't really tear every LLVM module away from Clang since the latter is built on the former set.
As for the reason why Clang is said to be a "compilation driver": Clang manages interpreting the command line parameters (descriptions and many declarations are TableGen'd and they might require a bit more than a simple grep to swim through the sources), decides which Jobs and phases are to be executed, set up the CodeGenOptions according to the desired/possible optimization and transformation levels and invokes the appropriate modules (clangCodeGen in BackendUtil.cpp is the one that populates a module pass manager with the optimizations to apply) and tools (e.g. the Windows ld linker). It steers the compilation process from the very beginning to the end.
Finally I would suggest reading Clang and LLVM documentation, they're pretty explicative and most of your questions should look for an answer there in the first place.
It's not exactly like GCC, so don't spend too much time trying to match the two precisely.
The LLVM compiler is a compiler for one specific language, LLVM. What Clang does is compile C++ code to LLVM, without optimizations. Clang can then invoke the LLVM compiler to compile that LLVM code to optimized assembly.
I have written my lexer and parser in flex and bison. My project is based on C++ and I would love to stick to it. Currently my Interpreter is written in C++ but I want to achieve faster execution time by converting to bytecode (some form of a VM-level bytecode) when my interpreter works. I know this can be achieved through LLVM. I had problems using it from a x64 OS and developing on a Visual Studio 2012 (32-bit). Some of which can be found # LLVM linker errors on VS. The other tool I came across is ANTLR and if I understand correctly then the latest release does not easily integrate into C++ yet. Many references were found for the same but a quick one can be # ANTLR integration with C++ issue. Also I do not want to dispose off my lexer and parser written in flex and bison. What are my options if I want to generate bytecode from my AST?
EDIT: My aim is to generate bytecode from my AST (for the target architecture) so the code can be executed at a Virtual Machine level. Currently I have an Interpretor which interpretes (executes the AST) based on C++ library and generates bytecode. I want to generate Bytecode straight from my AST and execute the AST in its bytecode.
Would be appreciated.
Generating native bytecode directly from your AST is not possible (well actually it is, but that would be extremely difficult). You need some kind of intermediary step like emitting LLVM bytecode or code in some programming language of your choice. Please note that LLVM bytecode is not the same as native target machine bytecode. The LLVM bytecode has to be compiled to native binaries for target machines which is done by the respective frontend. So you could as well just generate C++ code from your AST using a handwritten code emitter which traverses your syntax tree. Then you use a C++ compiler for the target platform to compile it to the desired native binary.
I have LLVM IR code in text format. What I wanna do is to be able to parse it and modify that code. Is there an API which can help in parsing the LLVM IR code? What libraries should I have in my system? At this moment I have clang compiler installed as well LLVM, as I can use commands such as llc, opt and llvm-link.
LLVM is primarily a C++ library. It has all the tools you can imagine to parse, manipulate and produce IR in both textual and bitcode (binary) formats.
To get started, take a look at the llvm::ParseIRFile function, defined in header include/llvm/Support/IRReader.h.
The best way to proceed would be to download the LLVM source code and build it, following these instructions. It's then easy to write your own code that uses the LLVM libraries.
How does one generate executable binaries from the c++ side of LLVM?
I'm currently writing a toy compiler, and I'm not quite sure how to do the final step of creating an executable from the IR.
The only solution I currently see is to write out the bitcode and then call llc using system or the like. Is there a way to do this from the c++ interface instead?
This seems like it would be a common question, but I can't find anything on it.
LLVM does not ship the linker necessary to perform this task. It can only write out as assembler and then invoke the system linker to deal with it. You can see the source code of llvm-ld to see how it's done.
The LLVM Core project consists of:
Compiler - converts source code to LLVM IR
VM - executes compiled IR code
How can I embed the VM to a C++ application?
The LLVM is really a collection of libraries that you can link to, so it's pretty easy to embed. More often the LLVM takes IR that you generate and compiles it directly to machine code. There is also a library available to interpret and execute IR for platforms that do not support JIT compilation.
There's a pretty good tutorial available on the LLVM website here: http://llvm.org/docs/tutorial/. I suggest that you go through that and then ask more specific questions if you have them.
Take a look at the HowToUseJIT example in LLVM.