How to use clang and distcc to compile on a slave of a different architecture (e.g. Mac/Linux) - c++

I want to use distcc to compile code from my Mac to a bunch of Linux hosts, but I can't figure out how to make everything "line up". I've successfully used distcc from Mac to Mac, so I have a general idea of how to set things up.
I'm using Clang 4.0 and have it installed and working on both Mac and Linux. The following command compiles fine without the distcc at the beginning, but after adding distcc, I get the following issues:
distcc /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/clang++ -I/usr/local/include -I/Users/xaxxon/v8toolkit/./include -I/Users/xaxxon/v8/include -stdlib=libc++ -g -Werror=return-type -g -std=gnu++1z -o CMakeFiles/v8toolkit_static.dir/src/v8toolkit.cpp.o -c /Users/xaxxon/v8toolkit/src/v8toolkit.cpp
I'm not sure what compiler is being picked up on Linux, nor do I know how to find out. It's possible it's picking up GCC instead of Clang.
My first concern is this:
clang: warning: argument unused during compilation: '-stdlib=libc++'
My first error is:
In file included from /Users/xaxxon/v8toolkit/src/v8toolkit.cpp:5:
In file included from /usr/include/assert.h:44:
In file included from /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/stdlib.h:94:
/usr/include/stdlib.h:250:20: error: blocks support disabled - compile with -fblocks or pick a deployment target that supports them
int atexit_b(void (^)(void)) __attribute__((availability(macosx,introduced=10.6)));
The next error I get (which becomes the first error if I manually add -fblocks to the compilation command (which isn't needed on a native Mac build) is:
/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:289:13: error: unknown type name '__type_pack_element'
typedef __type_pack_element<_Ip, _Types...> type;
^
/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:289:32: error: expected member name or ';' after declaration specifiers
typedef __type_pack_element<_Ip, _Types...> type;
~~~~~~~~~~~~~~~~~~~~~~~~~~~^
/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:356:43: error: use of undeclared identifier '__type_pack_element'
typename _ApplyFn::template __apply<__type_pack_element<_Idx, _Types...>>...
^
/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:357:6: error: expected a type
>;
I don't understand if I'm doing something fundamentally wrong or if there's something small I'm missing which is making the Linux compiler act differently.
I just made sure to have Clang on Linux in the same named directory and now am only getting the -fblocks and unused during compilation -stdlib=libc++ issues.
I can get everything to compile (albeit with warnings), but when it links, I get:
ld: warning: ignoring file CMakeFiles/v8toolkit_shared.dir/src/debugger.cpp.o, file was built for
unsupported file format ( 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 0x00 0x00
0x00 0x00 0x00 0x00 0x00 0x00 ) which is not the architecture being linked
(x86_64): CMakeFiles/v8toolkit_shared.dir/src/debugger.cpp.o

As long as you specify the -target flag, you can even cross-compile, because Clang is a native cross compiler.
Cross-compilation using Clang

Adding the -target flag fixes everything! In my case, for Mac OS X v10.11 (El Capitan), the target triple is:
-target x86_64-apple-darwin15.6.0
for a build command of:
distcc /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/clang++ -Dv8toolkit_shared_EXPORTS -I/usr/local/include -I/Users/xaxxon/v8toolkit/./include -isystem /Users/xaxxon/v8/include -stdlib=libc++ -g -Werror=return-type -target x86_64-apple-darwin15.6.0 -g -fPIC -std=gnu++1z -o CMakeFiles/v8toolkit_shared.dir/src/v8toolkit.cpp.o -c /Users/xaxxon/v8toolkit/src/v8toolkit.cpp
You can get the current host's target by typing clang -v:
$ clang -v
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-apple-darwin15.6.0 <<==== THIS LINE HERE
Thread model: posix
InstalledDir: /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin
The following CMake lines will grab (and print) the triple for the current machine:
# Get the target triple for the current host by calling clang -v and then stripping out the Target: value from its output. CMake regex syntax is quite limited.
execute_process(COMMAND ${CMAKE_CXX_COMPILER} -v ERROR_VARIABLE CLANG_VERSION_INFO)
string(REGEX REPLACE ".*Target:[\r\n\t ]*([^\r\n\t]*).*Thread model.*" "\\1" TARGET_TRIPLE ${CLANG_VERSION_INFO})
message(STATUS "TARGET TRIPLE: '${TARGET_TRIPLE}' END")
Much thanks to #duskwuff and oftc.net #llvm for help figuring this out!

Related

Controlling scons environment checking options

I'm trying to build mongodb (open source version 4.2) which uses python and scons for building. The problem relates to scons rather than mongodb.
My build fails very early with Couldn't find OpenSSL crypto.h header and library. Verbose details are:
file /.../SConstruct,line 3042:
Configure(confdir = build/scons/opt/sconf_temp)
scons: Configure: Checking for SSLeay_version(0) in C library crypto...
build/scons/opt/sconf_temp/conftest_d6743137aeb7fb2674cc9632f9989034_0.c <-
|
|
|#include "openssl/crypto.h"
|
|int
|main() {
| SSLeay_version(0);
|return 0;
|}
|
gcc -o build/scons/opt/sconf_temp/conftest_d6743137aeb7fb2674cc9632f9989034_0.o -c -std=c11 -ffp-contract=off -fno-omit-frame-pointer -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -Wno-exceptions -fstack-protector-strong -fno-builtin-memcmp -fPIE -DNDEBUG -D_XOPEN_SOURCE=700 -D_GNU_SOURCE build/scons/opt/sconf_temp/conftest_d6743137aeb7fb2674cc9632f9989034_0.c
cc1: error: command-line option '-Wno-exceptions' is valid for C++/ObjC++ but not for C [-Werror]
cc1: all warnings being treated as errors
scons: Configure: no
I'm using Arch Linux which has multiple OpenSSL packages, and the default 3.0 is not compatible with mongodb source. I also have OpenSSL 1.1 and 1.0 installed, and can switch with e.g. gcc -I/usr/include/openssl-1.1.
Unfortunately I have not found a way to instruct SConstruct to use this flag in the command lines it generates to check the environment. I have tried CFLAGS, CCFLAGS, CPPPATH both as environment variables and scons command line parameters.
I also tried reverse enginering it, and tracked this to Conftest and TryBuild but it's not obvious how I can influance theses from the command line, so I'm trying my luck with you guys before going deeper in scons code.
If you look at the SConstruct and search for openssl, you'll find this blurb under the logic to detect on macOS
NOTE: Recent versions of macOS no longer ship headers for the system OpenSSL libraries.
NOTE: Either build without the --ssl flag, or describe how to find OpenSSL.
NOTE: Set the include path for the OpenSSL headers with the CPPPATH SCons variable.
NOTE: Set the library path for OpenSSL libraries with the LIBPATH SCons variable.
NOTE: If you are using HomeBrew, and have installed OpenSSL, this might look like:
\tscons CPPPATH=/usr/local/opt/openssl/include LIBPATH=/usr/local/opt/openssl/lib ...
NOTE: Consult the output of 'brew info openssl' for details on the correct paths."""
I'd bet if you did the same but pointed at the proper locations on your system for the openssl libs and header files, you'd be able to build.
(see: https://github.com/mongodb/mongo/blob/r4.2.0/SConstruct#L3015 )

c++ / g++ - Error: Unknown pseudo-op: `. loc'

I am trying to configure my computer to learn C++.
I have errors and warnings once I run gcc from the terminal to call hello.cpp.
C:\Users\Mathieu>gcc -g hello.cpp -o hello -lm
I am using a Windows computer / windows 7 (64 bits) / IDE: Sublime text or Visual Studio Code.
What I have done so far by looking for solutions in the web:
I have added into the PATH the address of MinGW and I have added ";" before the previous PATH.
I have also been checking all the PATH by using echo %PATH% into the terminal and checking the environmental variables.
By typing g++ or gcc alone in the terminal, I get "fatal error, no input file", which means gcc and g++ are well detected by the computer.
I have checked g++ and gcc versions, there are equal.
gcc (MinGW.org GCC Build-20200227-1) 9.2.0
gcc (MinGW.org GCC Build-20200227-1) 9.2.0
I have also performed #alisonc recommandation found here by reviewing links.
g++ 4.6.1 compiler error: Error: unknown pseudo-op: `.cfi_personality'
C:\Users\Mathieu>ls 'which g++' -al
ls: which g++: No such file or directory
C:\Users\Mathieu>ls 'which gcc' -al
ls: which gcc: No such file or directory
When I have launched the program from the terminal I get a long list of errors. I am just adding 3 lines to illustrate. I am not sure if that is relevant to add everything, please, see a few ones:
C:\Users\Mathieu\AppData\Local\Temp\ccUFD66I.s:6498: Error: Unknown pseudo-op:
.secrel32
C:\Users\Mathieu\AppData\Local\Temp\ccUFD66I.s:153: Warning: Missing string
C:\Users\Mathieu\AppData\Local\Temp\ccUFD66I.s:153: Error: Rest of line ignored.
First ignored character is `3'.'''
May someone help me ?
Thank you in advance
All the best
Mathieu
Try to run:
g++ -g hello.cpp -o hello -lm

Unable to cross compile Linux openGL program with mingw32. Config.log error: undefined reference to `_glEnable'

I am attempting to cross-compile an OpenGL program using Mingw32 but have run into a road block. After invoking mingw32-configure; the compilation is interrupted by
configure: error: lacking proper OpenGL support
I checked the config.log file and found the following entries:
configure:21709: checking GL/gl.h usability
configure:21726: ccache i686-pc-mingw32-g++ -c -O2 -g -pipe -Wall -,-D_FORTIFY_SOURCE=2 -fexceptions --param=ssp-buffer-size=4 -mms-bitfields conftest.cpp >&5
configure:21732: $? = 0
configure:21746: result: yes
configure:21750: checking GL/gl.h presence
configure:21765: ccache i686-pc-mingw32-g++ -E conftest.cpp
configure:21771: $? = 0
configure:21785: result: yes
configure:21813: checking for GL/gl.h
configure:21820: result: yes
configure:21834: checking for glEnable in -lGL
configure:21869: ccache i686-pc-mingw32-g++ -o conftest.exe -O2 -g -pipe - Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions --param=ssp-buffer-size=4 -mms- bitfields conftest.cpp -lGL >&5
/tmp/ccjGmlvX.o: In function 'main':../rpmbuild/SOURCES/poker3d-1.1.36/conftest.cpp:34: undefined reference to `_glEnable'
collect2: ld returned 1 exit status
configure:21875: $? = 1
configure: failed program was:
| /* confdefs.h. */
I have added the llvm software rasterizer to my system after a suggestion on this website from another thread related to this problem and have also implemented a suggestion from the Mingw site to copy libopengl32.a to libGL.a. So far nothing has changed the errors that I get after each compile attempt.
Give me some advice on how to resolve this.
autotools/autoconf is abysmally bad for crosscompilation. The way the generated configure script works is, that for many tests it attempts to compile and execute short test snippet programs. Of course the whole "execute" part will miserably fail for cross compiled libraries, due to either lack in instruction set support (compiling for a different CPU) or targeting an entirely different OS.
At least in Linux one can work around this using the "misc binfmt" kernel support, which allows to register interpreters for non-native executable formats. For Windows targets you'd use Wine as an interpreter and for foreign CPUs you can use QEmu which not only can emulate whole machines, but focus CPU emulation on a single process/binary. And of course you can combine it.
However this is just a crutch and honestly you should probably ditch autoconf/autotools entirely and just write Makefiles. Today the *nix-ish systems are so similar that none of these compiled tests make a lot of sense at all.

host_config.h:unsupported GNU version! gcc versions later than 4.9 are not supported [duplicate]

I struggle with Caffe compilation. Unfortunately I failed to compile it.
Steps I followed:
git clone https://github.com/BVLC/caffe.git
cd caffe
mkdir build
cd build
cmake ..
make all
Running make all fails with the following error message:
[ 2%] Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/util/cuda_compile_generated_im2col.cu.o
In file included from /usr/include/cuda_runtime.h:59:0,
from <command-line>:0:
/usr/include/host_config.h:82:2: error: #error -- unsupported GNU version! gcc 4.9 and up are not supported!
#error -- unsupported GNU version! gcc 4.9 and up are not supported!
^
CMake Error at cuda_compile_generated_im2col.cu.o.cmake:207 (message):
Error generating /mydir/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/util/./cuda_compile_generated_im2col.cu.o
Software version:
OS: Debian.
gcc version: 5.3.1.
nvcc version: 6.5.12.
cat /proc/driver/nvidia/version result:
NVRM version: NVIDIA UNIX x86_64 Kernel Module 352.63 Sat Nov 7 21:25:42 PST 2015
GCC version: gcc version 4.8.5 (Debian 4.8.5-3)
Attempts to solve the problem
1st try
Simple solutions are often best ones, so (as suggested here) I tried to comment out macro checking gcc version from /usr/include/host_config.h (line 82). Unfortunately it doesn't work and compilation fails badly:
1 catastrophic error detected in the compilation of "/tmp/tmpxft_000069c2_00000000-4_im2col.cpp4.ii".
2nd try
I tried to run:
cmake -D CMAKE_CXX_COMPILER=g++-4.8 ..
make
but it fails with exactly the same error message (even though g++-4.8 should be accepted).
3rd try
I've found similar problem (though not related to Caffe) and I tried to solve it as suggested in the accepted answer.
What I did:
I've ran grep -iR "find_package(CUDA" caffe command and found Cuda.cmake file which has find_package(CUDA 5.5 QUIET) in line 225.
I added set(CUDA_HOST_COMPILER /usr/bin/gcc-4.8) to Cuda.cmake, line before line: find_package(CUDA 5.5 QUIET).
I removed everything from build directory and ran cmake and make again - with and without -D CMAKE_CXX_COMPILER=g++-4.8.
Unfortunately result is exactly the same. Caffe probably overwrites it somehow - I didn't figure it out how.
make VERBOSE=1 2>&1 | grep -i compiler-bindir returns nothing.
What's interesting, make VERBOSE=1 prints command that fails, which is:
/usr/bin/nvcc -M -D__CUDACC__ /mydir/caffe/src/caffe/util/im2col.cu -o /mydir/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/util/cuda_compile_generated_im2col.cu.o.NVCC-depend -ccbin /usr/bin/cc -m64 -DUSE_LMDB -DUSE_LEVELDB -DUSE_OPENCV -DWITH_PYTHON_LAYER -DGTEST_USE_OWN_TR1_TUPLE -Xcompiler ,\"-fPIC\",\"-Wall\",\"-Wno-sign-compare\",\"-Wno-uninitialized\",\"-O3\",\"-DNDEBUG\" -gencode arch=compute_20,code=sm_21 -Xcudafe --diag_suppress=cc_clobber_ignored -Xcudafe --diag_suppress=integer_sign_change -Xcudafe --diag_suppress=useless_using_declaration -Xcudafe --diag_suppress=set_but_not_used -Xcompiler -fPIC -DNVCC -I/usr/include -I/mydir/caffe/src -I/usr/include -I/mydir/caffe/build/include -I/usr/include/hdf5/serial -I/usr/include/opencv -I/usr/include/atlas -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/mydir/caffe/include -I/mydir/caffe/build
when I add --compiler-bindir /usr/bin/gcc-4.8 flag manually, it prints error:
nvcc fatal : redefinition of argument 'compiler-bindir'
which may be related to this bug report.
Edit: I didn't notice that --compiler-bindir and -ccbin are the same options, and the latter is already set in above command that failed. When I changed -ccbin /usr/bin/cc to -ccbin /usr/bin/gcc-4.8 in above command that failed, it completes successfully. Now I need to find option in Caffe's CMake file that overwrite -ccbin in all subsequent Caffe's CMakes. Looking at cmake/Cuda.cmake:252:list(APPEND CUDA_NVCC_FLAGS ${NVCC_FLAGS_EXTRA} seems to be good way to go.
How can I successfully complete my compilation? Any help is appreciated.
Related SO questions:
host_config.h:unsupported GNU version! gcc versions later than 4.9 are not supported.
CUDA 6.5 complains about not supporting gcc 4.9 - what to do?.
cmake -D CUDA_NVCC_FLAGS="-ccbin gcc-4.8" .. && make causes successful compilation.
Now another problem showed up: linking Google's libgflags or libprotobuf fails probably due to fact that it was compiled with newer gcc version but it's not related to asked question.
My machine runs Ubuntu 15.10, and my default compiler version is gcc 5.2.1 .
Commenting out the #error directive in line 115 of file
/usr/local/cuda-7.5/include/host_config.h
(or whatever the path on your system is) did the trick for me. Caffe compiled fine, all tests ran smoothly.
On the other hand, if one chooses to ignore this and proceed to compile part of the project with one compiler version, part of the project with another (for me it was gcc-4.8 and gcc-5.2.1), linking problems will arise. The linking problems of protobuf and libgflags another answer mentions are not unrelated to this.

cc1plus: error: unrecognized command line option "-fsysroot=<path>" with g++

Am trying to compile a large C++ project in a simulated 32bits environment (cel5.03-i386-2.3) on a 64bits machine. (The compiling worked well on 32bits machine before)
During this compiling some files fail as below. The weird thing is that the argument passed to g++ command is "--sysroot=" while in error message it becomes "-fsysroot=".
Why could this happen?
/usr/bin/g++ -MMD -MF /proj/src/.debug/lib/osiris/core/utils/WWNConverter.d -I/proj/src/lib/osiris -DDEBUG_LOG -DDEBUG -D_REENTRANT -DAPE_MODE -DSWAT_MODE -g -Werror -Wall -Wno-write-strings -fPIC -m32 --sysroot=/auto/andpkg/rep_cache//wr-x86/3.0FCS/sysroot -rdynamic -I/proj/src -I/proj/src/.debug/include/private -I/proj/src/.debug/include/public -I/proj/src/lib/framework -I/proj/src/lib/osiris -I/proj/src/lib/prt -I/proj/src/lib/callhome -I/proj/src/lib/snmp -I/proj/src/.debug/external/pcre/include -I/proj/src/external/sqlite/include -I/proj/src/external/openssl/openssl-fips-1.2.3/include -I/proj/src/support/storage-driver/include -I/proj/src/external/json/include -DTARGET_OBJECT=libosiris.so -I/proj/src/.debug/external/pcre/include -I/proj/src/.debug/external/libcurl/include -o /proj/src/.debug/lib/osiris/core/utils/WWNConverter.o -c /proj/src/lib/osiris/core/utils/WWNConverter.cc
make[1]: *** [/proj/src/.debug/lib/osiris/core/utils/WWNConverter.o] Error 1
cc1plus: error: unrecognized command line option "-fsysroot=/auto/andpkg/rep_cache//wr-x86/3.0FCS/sysroot"
The 32-bit compiler (GCC 3.4.6) is too old to support the --sysroot option, which was added to trunk with commit 102367, and did not make it into a release branch until GCC 4.1.0.
2005-07-25 Mark Mitchell <mark#codesourcery.com>
* gcc.c (option_map): Add --sysroot.
(process_command): Handle --sysroot.
(display_help): Document it.
* doc/cppopts.tex (-isysroot): Document.
* doc/invoke.texi (--sysroot): Document.
* doc/install.texi (--with-build-sysroot): Document.
* Makefile.in (inhibit_libc): New variable.
(INHIBIT_LIBC_CFLAGS): Likewise.
(LIBGCC2_CFLAGS): Include
$(INHIBIT_LIBC_CFLAGS).
(CRTSTUFF_CFLAGS): Include $(INHIBIT_LIBC_CFLAGS).
($(T)crtbegin.o): Do not use #inhibit_libc#.
($(T)crtend.o): Likewise.
($(T)crtbeginS.o): Do not use #inhibit_libc#.
($(T)crtendS.o): Likewise.
($(T)crtbeginT.o): Do not use #inhibit_libc#.
($(T)crtendT.o): Likewise.
(stmp-fixinc): Do not complain about missing headers if
inhibit_libc.
* configure.ac (inhibit_libc): Set it to true/false.
(--with-build-sysroot): New option. Use it to set
SYSTEM_HEADER_DIR.
* configure: Regenerated.