I am working in a project that consist of some C++ teams. Each team delivers libraries and our team is integrating these libraries into a front end application.
The application is cross platform, so it means that other the teams have to provide the same (static) libraries compiled for different platforms/CPU architecture/configuration. Eg. we have Visual Studio 2015/2013, 32bit/64bit, linux, Debug/Release etc.
It would be nice to reduce the number of these static library "manifests", so I was looking into the Clang/LLVM. The idea would be compile the static libraries into LLVM bitcode and use the llvm-ar tool to create an llvm static library. When we have to make the binaries for a specific platform we would use the llc (LLVM platform compiler) to create the native code static library and do the linking with the platform linker.
Questions:
is there a better way to do what I want to achieve?
the llc does not seem to support the compiling of a static library, only individual translation units (.bc -> .o). Of course I can extract each individual bitcode file, assemble it to native object file and use the platform librarian tool (lib/ar) to make the static library, but I wonder if there is a more streamlined solution.
the gold linker seems to make something I need, but seems to be restricted to ELF format. I have to support Windows/Linux and maybe IOS
LLVM IR generated from target-specific and platform-specific language (C/C++) won't be target neutral. Think about type sizes, alignments, ABI requirements, etc. Not the mention pure source code features like preprocessor. So, no, the approach you thought about won't work at all.
See LLVM bitcode cross-platform for some more information.
Related
I'm writing a compiler that embeds the LLVM API. By copying code from the llc tool, I can output assembly language or object files that I can turn into binaries using clang or an assembler.
But I want my compiler to be self contained. Is it possible to turn LLIR into binaries using LLVM? This seems like the sort of thing that should be in the LLVM toolkit.
Yes, it is possible and this is also done by llc with -filetype=obj argument.
You can consult the compileModule function to learn how to use the programmatic API.
Note that this will only generate an object file for a given translation unit. You will also need a linker to convert it into a proper executable or library. The LLVM linker, lld, can also be embedded into client applications as a library, so in the end you will be able to create a self-hosting compiler.
I am trying to compile various programs such as MariaDB with a musl toolchain. That is, I don't want any dependencies on glibc or GNU's linker after compilation has finished.
Thus far, I have been using musl's GCC wrapper, musl-gcc to compile things. But, with larger programs like MariaDB I struggle to get all the necessary libraries and headers and symlinking or adding environment variables for the compilation doesn't really help.
I see mention of building a cross-compiler targeting musl libc with additional documentation and code at this GitHub repo. From the documentation on the cross-compiler:
This gives you a full, relocatable musl-targeting toolchain, with C++ support and its own library paths you can install third-party libraries into.
It sounds like this could help me, but I am not sure how this is very different from musl's GCC wrapper, which as I understand, just alters where GCC looks for libraries and headers etc.
Ultimately, I am unsure how different this cross-compiler really is from the GCC wrapper and if it would be useful in my case. Why would I need my own library paths to install third-party libraries into when I can just symlink to existing libraries and use the GCC wrapper? Is the cross-compiler the way I should be compiling things, especially bigger code bases?
All the wrapper does is remove the default library and include paths and add replacement ones. Otherwise, it relies on the compiler's view of the ABI being sufficiently matched that a GCC that thinks it's targeting glibc (*-linux-gnu) works with a musl (*-linux-musl) target.
A full GCC toolchain has a number of "target libraries" - libraries that are linked into the output program to provide some functionality the compiler offers. This includes libgcc (software multiply or divide, software floating point, etc. according to whether the target arch needs these things, and unwinding support for exception handling), libstd++ (the C++ standard library), and a number of other things like libgomp (GNU OpenMP runtime implementation for use with #pragma OMP ...). In theory all of these possibly depend on the specific target ABI, including the libc, and potentially link to symbols from libc. In practice, for the most part libgcc doesn't, and is "safely" reusable with a different libc as long as the basic type definitions and a few other ABI things match.
For the other libraries with more complex dependencies on libc, it's generally not expected that a build against one libc would work with a different one. musl libc provides some degree of ABI-compat for using glibc-linked libraries, so there's some hope that they'd work anyway, but you'd need to copy them to the new library path. However, GCC's C++ standard library headers also seem to have some heavy dependency on the (libc) system headers for the target, and when setup for glibc do not seem to work with musl. There is probably some way to make this work (i.e. to add C++ support to the wrapper), but it's an open problem exactly how to do it.
Even if this could be made to work, though, the deeper you go, the more reasons you're likely to find that you'd rather just have a proper cross-compiler toolchain that knows it's targeting musl. And nowadays these are easy enough to build. But if you find you're needing additonal third party libraries too, and don't want to build them yourself, it probably makes more sense to just use a Docker image (or other container) with a distro that provides library binaries for you.
I'm searching for a C or C++ library which can load and link obj files (doesn't matter if ELF or obj) dynamicly at runtime. I spend some time searching for such library, but my results weren't successful.
What I tried:
LLVM:
Currently my best solution! I used Clang to generate .obj files in the bytecode format of LLVM and used its JIT functions to dynamic load and execute the function. But, the LLVM is huge and my PC at home hasn't the power to compile the complete LLVM just for the JIT. Also I encountered some problems with relocation overflows or not implemented relocation types.
libjit:
I read, that it can load .elf files and link them too. But sadly, I couldn't compile it for windows, so I couldn't try.
Nanojit and NativeJit:
It seems like they don't support JITting an object file.
So... What can I do? Do I have to stick around with the LLVM? Are there any alternatives?
I suppose that an analogy that can be taken as a 1st approach is that the .bc is similar to an .o (or .obj) file in that it is just the translation of C++ code to an intermediate language, and tht it can contain references to functions not defined in it, to be searched in libraries.
And that the JIT-ted code is similar to a DLL, in the sense that it will be linked dynamically to the executable where it will run in.
You need not to compile LLVM -- you can download the binaries for LLVM and assorted utilities (like clang) from LLVM Download Page
I'm trying out the clang32 compiler coming with C++ builder 10.2. Builder don't yet have any good support for CMake, so a great number of 3rd party libraries are (very) hard to compile using it.
Anyone knowing if there is any binary compatibility between clang32 and Visual Studio compiler?
There are essentially three different levels of compatibility you need to worry about:
File formats for object code and debug data, which allow you to use clang to build part of your project and Visual C++ cl.exe to build a library and then link them together and debug both.
Ability to write code and structure data that conforms to a portable binary interface, so it can be called across a mix of compilers.
Binary compatibility of the C++ standard library, so that standard library objects can be shared across a mix of compilers.
I can definitely say that (2) is supported and (3) absolutely is not; you can't even share standard library objects between different patchlevels of the same compiler. For (1) I don't know. A common way to bypass the issues with (1) is to build a DLL using each compiler, so they dynamically interface but no static linking nor merging of debug data is necessary.
If your concern is about cmake though, I think your problem is not the compiler (clang is available for Linux and cmake supports it well -- you should find the make scripts are capable of configuring all the compiler options). Whether it can generate project files for C++Builder is a different story, but perhaps you should consider using a different IDE. There are many with clang support, even Microsoft's Visual Studio has some ability to use clang for the compile step, and it's getting better with each release.
I'm creating a library in C++. It links against Windows libraries on Windows and Linux libraries on Linux. It's abstracted, all is well.
However, is it feasible to dynamically detect, load and use libraries (and copying header files for use) so it could be used on any platform if it was running under LLVM JIT?
Unfortunately, the LLVM intermediate representation in the bitcode files is not machine completely machine independent. You could probably get away with x86 Linux and Windows, but that same bitcode would probably not run on x86_64 systems, for example.