In LLVM, when a CUDA source file is compiled into PTX file, opcodes are generated. Now which section on LLVM converts these opcodes in PTX file to binary (hexcode) (i.e the final executable). Searching the llvm/lib/Target/NVPTX didn't give an answer to this.
Normally this would be in NVPTXInstrInfo.td though it looks like they have assembly mnemonics specified but not encoding bits. Usually this means LLVM isn't doing the assembling and this is instead passed to an external assembler.
You can look at HexagonInstrInfo.td to see what encoding bits would look like. Search for "let Inst{"
Related
I'm doing a little project using the Android NDK, and I must insert some asm code for ARM architecture.
Everything apart from the asm works just fine, but the asm code tells me that
Operand 1 should be an integer register
when conpiling simple code like
asm("mov r0, r0");
So, what is the problem? Is my computer trying to compile for x86_64 instead of ARM? If so, how should I change that?
Also, I've tried the x86_64 equivalent arm("mov rax, rax"); but the error is the same.
All your C sources are compiled for each architecture that is mentioned in your APP_ABI. So there is no point to wonder why ARM assembly was not unterstood by x64 compiler or vice versa.
Do not use inline assembly. It is much better to put all assembly stuff into dedicates *.S sources, that will be processed by as (NDK toolchains have it). That assembly sources should be placed into appropriate folders like arch-arm/, arch-x86/. Then you should add them to Android.mk properly:
LOCAL_SRC_FILES := arch-$(TARGET_ARCH)/my_awesome_code.S
$(TARGET_ARCH) helps to resolve path to appropriate source in correct and painless way.
P.S. Also standalone assembly gives more abilities to you than inline one. This is one more reason to avoid using of inline assembly. Moreover inline assembly syntax differs for compiler to compiler since it is not a part of standard.
I'm currently working on a compiler project using llvm. I have followed various tutorials to the point where I have a parser to create a syntax tree and then the tree is converted into an llvm Module using the provided IRBuilder.
My goal is to create an executable, and I am confused as what to do next. All the tutorials I've found just create the llvm module and print out the assembly using Module.dump(). Additionally, the only documentation I can find is for llvm developers, and not end users of the project.
If I want to generate machine code, what are the next steps? The llvm-mc project looks like it may do what I want, but I can't find any sort of documentation on it.
Perhaps I'm expecting llvm to do something that it doesn't. My expectation is that I can build a Module, then there would be an API that I can call with the Module and a target triple and an object file will be produced. I have found documentation and examples on producing a JIT, and I am not interested in that. I am looking for how to produce compiled binaries.
I am working on OS X, if that has any impact.
Use llc -filetype=obj to emit a linkable object file from your IR. You can look at the code of llc to see the LLVM API calls it makes to emit such code. At least for Mac OS X and Linux, the objects emitted in such a manner should be pretty good (i.e. this is not a "alpha quality" option by now).
LLVM does not contain a linker (yet!), however. So to actually link this object file into some executable or shared library, you will need to use the system linker. Note that even if you have an executable consisting of a single object file, the latter has to be linked anyway. Developers in the LLVM community are working on a real linker for LLVM, called lld. You can visit its page or search the mailing list archives to follow its progress.
As you can read on the llc guide, it is indeed intended to just generate the assembly, and then "The assembly language output can then be passed through a native assembler and linker to generate a native executable" - e.g. the gnu assembler (as) and linker (ld).
So the main answer here is to use native tools for assembling and linking.
However, there's experimental support for generating the native object directly from an IR file, via llc:
-filetype - Choose a file type (not all types are supported by all targets):
=asm - Emit an assembly ('.s') file
=obj - Emit a native object ('.o') file [experimental]
Or you can use llvm-mc to assemble it from the .s file:
-filetype - Choose an output file type:
=asm - Emit an assembly ('.s') file
=null - Don't emit anything (for timing purposes)
=obj - Emit a native object ('.o') file
I don't know about linkers, though.
In addition, I recommend checking out the tools/bugpoint/ToolRunner.h file, which exposes a wrapper combining llc and the platform's native C toolchain for generating machine code. From its header comment:
This file exposes an abstraction around a platform C compiler, used to compile C and assembly code.
Check out these functions in llvm-c/TargetMachine.h:
/** Emits an asm or object file for the given module to the filename. This
wraps several c++ only classes (among them a file stream). Returns any
error in ErrorMessage. Use LLVMDisposeMessage to dispose the message. */
LLVMBool LLVMTargetMachineEmitToFile(LLVMTargetMachineRef T, LLVMModuleRef M,
char *Filename, LLVMCodeGenFileType codegen, char **ErrorMessage);
/** Compile the LLVM IR stored in \p M and store the result in \p OutMemBuf. */
LLVMBool LLVMTargetMachineEmitToMemoryBuffer(LLVMTargetMachineRef T, LLVMModuleRef M,
LLVMCodeGenFileType codegen, char** ErrorMessage, LLVMMemoryBufferRef *OutMemBuf);
To run the example BrainF program, compile it and run:
echo ,. > test.bf
./BrainF test.bf -o test.bc
llc -filetype=obj test.bc
gcc test.o -o a.out
./a.out
then type a single letter and press Enter. It should echo that letter back to you. (That's what ,. does.)
The above was tested with LLVM version 3.5.0.
I'm linking two modules together (in KLEE, but that isn't important, I think) and llvm complains because they seem to have different data layouts:
WARNING: Linking two modules of different data layouts!
I had a look at those and it turns out the first libkleeRuntimeIntrinsic.bca has the following layout
e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64
while my own client.bc has
e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-f128:128:128-n8:16:32:64
I compared these and the latter has a f128:128:128 too much, which according to the documentation indicates support for 128 bit floats.
These were built on the very same system with the same version of llvm-gcc and linked/archieved with llvm-ld/llvm-ar. All llvm version 2.9 (my llvm is configured in Debug+Asserts mode).
What is the reason for this, and how can I get the second (client.bc) to have the same daya layout (that is, without the f128:128:128)?
Here I have an executable without knowing its build environment, with the assumption of gcc/g++ being used.
Is there a way to find out the optimization flag used during compilation (like O0, O2, ...)?
All means are welcomed, no matter it's by analyzing the binary or some debug test via gdb (if we assume that -g flag is available during compilation).
If you are lucky, the command-line is present in the executable file itself, depending on the operating system and file format used. If it is an Elf-file, try to dump the content using the objdump from GNU binutils
I really don't know if this can help, but checking O0 is quite easy in the disassembly (objdump -d), since the generated code has no optimization at all and adds some few instructions to simplify debugging.
Typically on an x86, the prologue of a function includes saving the stack pointer (for the backtrace, I presume). So if you locate, for example, the main function, you should see something like:
... main:
... push %rbp
... mov %rsp,%rbp
And you should see this "pattern" at almost every beginning of the functions.
For other targets (I dunno what your target platform is), you should see more or less similar assembly sequences in the prologues or before the function calls.
For other optimization levels, things are way way trickier.
Sorry for remaining vague and not answering the entire question... Just hoping it will help.
Is it possible to compile program on one platform and link with other ? What does object file contain ? Can we delink an executable to produce object file ?
No. In general object file formats might be the same, e.g. ELF, but the contents of the object files will vary from system to system.
An object file contains stuff like:
Object code that implements the desired functionality
A symbol table that can be used to resolve references
Relocation information to allow the linker to locate the object code in memory
Debugging information
The object code is usually not only processor specific, but also OS specific if, for example, it contains system calls.
Edit:
Is it possible to compile program on one platform and link with other ?
Absolutely. If you use a cross-compiler. This compiler specifically targets a platform and generates object files (and programs) that are compatible with the target platform. So you can use an X86 Linux system, for example, to make programs for a powerpc or ARM based system using the appropriate cross compiler. I do it here.
Is it possible to compile program on one platform and link with other ?
In general, no. Object files are compiler specific. Some compilers spit out COFF, others spit out ELF, etc. On top of that, you have to worry calling conventions, system calls, etc. This is platform dependent.
What does object file contain ?
Symbol tables, code, relocation, linking and debugging information.
If what you're after is portability, then write portable C/C++ and let a platform-specific compliant compiler do the work.
In practice, no. There are several things that would have to be the same:
- OS interface (same system calls)
- memory layout of data (endianness, struct padding, etc.)
- calling convention
- object file format (e.g. ELF is pretty standard on Linux)
Lookup ABI for more information.
It doesn't need to be said again: C/C++ object files aren't portable.
On the other hand, ANSI C is one of the most portable languages there is. You may not be able to pick up your object files, but recompiling your source is likely to work if you stick to the ANSI C standard. This might also be true of C++ as well.
I don't know how universal GNU C++ is, but if you can compile with gcc on one computer you're good to go on any other machine that also has gcc installed. Just about every machine you can think of has a C compiler. That's portability.
No. They are not platform independent. Take for instance, the GNU C Compiler (gcc), that generates ELF binary files. Windows compilers (Borland, Microsoft, Open Watcom) can produce Windows Binary PE (Portable Executable) format. Novell binaries are NLM (Netware Loadabable Module) format.
These examples above of the different outputs which is compiler dependant, there is no way, a linker on a Windows platform, would know anything about ELF format nor NLM format, hence it is impossible to combine different formats to produce an executable that can run on any platform.
Take the Apple's Mac OSX (before the Intel chips were put in), they were running on the PowerPC platform, even if it has the GNU C Compiler, the binary is specifically for the PowerPC platform, if you were to take that binary and copy it onto a Linux platform, it will not run as a result of the differences in the instructions of the platform's microprocessor i.e. PowerPC.
Again, same principle would apply to the OS/390 mainframe system, a GNU C compiler that produces a binary for that platform will not run on an pre-Intel Apple Mac OSX.
Edit: To further clarify what an ELF format would look like see below, this was obtained by running objdump -s main.o under Linux.
main.o: file format elf32-i386
Contents of section .text:
0000 8d4c2404 83e4f0ff 71fc5589 e55183ec .L$.....q.U..Q..
0010 14894df4 a1000000 00a30000 0000a100 ..M.............
0020 000000a3 00000000 8b45f483 38010f8e .........E..8...
0030 9c000000 8b55f48b 420483c0 048b0083 .....U..B.......
0040 ec086800 00000050 e8fcffff ff83c410 ..h....P........
0050 a3000000 00a10000 000085c0 7520a100 ............u ..
0060 00000050 6a1f6a01 68040000 00e8fcff ...Pj.j.h.......
0070 ffff83c4 10c745f8 01000000 eb5a8b45 ......E......Z.E
0080 f4833802 7e218b55 f48b4204 83c0088b ..8.~!.U..B.....
0090 0083ec08 68240000 0050e8fc ffffff83 ....h$...P......
00a0 c410a300 000000a1 00000000 85c07520 ..............u
00b0 a1000000 00506a20 6a016828 000000e8 .....Pj j.h(....
00c0 fcffffff 83c410c7 45f80100 0000eb08 ........E.......
00d0 e8fcffff ff8945f8 8b45f88b 4dfcc98d ......E..E..M...
00e0 61fcc3 a..
Contents of section .rodata:
0000 72000000 4552524f 52202d20 63616e6e r...ERROR - cann
0010 6f74206f 70656e20 696e7075 74206669 ot open input fi
0020 6c650a00 77000000 4552524f 52202d20 le..w...ERROR -
0030 63616e6e 6f74206f 70656e20 6f757470 cannot open outp
0040 75742066 696c650a 00 ut file..
Contents of section .comment:
0000 00474343 3a202847 4e552920 342e322e .GCC: (GNU) 4.2.
0010 3400 4.
Now compare that to a PE format for a simple DLL
C:\Program Files\Microsoft Visual Studio 9.0\VC\bin>dumpbin /summary "C:\Documents and Settings\Tom\My Documents\Visual Studio 2008\Projects\SimpleLib\Release\SimpleLib.dll"
Microsoft (R) COFF/PE Dumper Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file C:\Documents and Settings\Tom\My Documents\Visual Studio 2008\Projects\SimpleLib\Release\SimpleLib.dll
File Type: DLL
Summary
1000 .data
1000 .rdata
1000 .reloc
1000 .rsrc
1000 .text
Notice the differences in the sections, under ELF, there is .bss, .text, .rodata and .comment, and is an ELF format for i386 processor.
Hope this helps,
Best regards,
Tom.
They are platform dependent. For example file-command prints out following:
$ file foo.o
foo.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
C++ has the additional detail that the names that it puts into an object file are typically 'mangled' to deal type safety for names that are overloaded. The methods used to mangle names are not part of the C++ standard (in fact, name mangling is an implementation detail that's not required at all if the vendor can come up with a different way to implement overloading). So even for the same platform target, you cannot count on being able to link object files from one compiler vendor to another.
There are times when a compiler vendor might change the name mangling scheme from one compiler version to another. For example, I believe there are versions of MSVC for which you can't reliably link C++ object files from an older version to a newer version.
Some platforms have the name mangling specified in an ABI standard for the platform (such as ARM which uses the name mangling specified in the generic C++ ABI that was originally developed for SVr4 on Itanium), but others don't (Windows). Even for the ARM, I'm not sure how interoperable the ABI standard makes linking C++ object files that were created by different compilers.
I just wanted to say that as long as they use the same processor architecture and object format, as well as calling convention(usually nowadays, the processor maker creates one), there are many chances for object files to work interchangeably.
However, even in C the compiler makes some assumptions about certain library functions like stack protection(that I know of) being present, which need not be the same on both platforms. in the case that such code is generated, the objects will not be directly compatible.
System calls are not really relevant as long as the systems share them at all as normally they are called through C wrappers in the standard libraries.
In the end this only applies to C and very similar OSes like Linux and the BSDs, but it can happen.
It's possible to compile with GCC and create an object file in ELF file format and convert the object file to work in Visual Studio. I have done this multiple times now.
There are three things you need to know to do this: the function calling convention, the object file format, and the function name mangling.
Function calling conventions: For 32-bit mode the function calling convention is easy: they are the same for Windows and Unix. For 64-bit mode Windows and Unix use different calling conventions. Therefore, in 64-bit mode you have to get the calling convention correct. You can either do this when you compile or from the object file itself. It's much easier to do this when you compile. To have GCC use the Windows calling convention use -mabi=ms. To do this from the object file you need a tool. Agner Fog's objconv tool can do this for some functions.
Objection file format: To convert the object file format you need a tool. I use Agner Fog's objconv tool for this. It can convert from several different object file formats. For example to convert ELF64 to COFF64 (PE32+) do objconv -fcoff64 foo.o foo.obj.
Function name mangling: Due to function overloading in C++ compilers mangle the functions names. The details for each compiler can be found in Agner Fog's manual calling convetions. GCC and Visual Studio mangle function names differently. To work around this proceed function defintions with extern "C"
If you get all three of these correct and you don't make any OS specific calls than you may be able to use your object files between compilers successfully. There are other problems that can occur of course. See the manual in objconv for more details. But so far this method had worked well for me.