How to use SYSTEM headers with CMake and clang-tidy? - c++

I am trying to use clang-tidy in my CMake (3.17.1) project however it crashes on the Catch2 test library header. Setting the Catch2 as a system header does not seem to help. The command invoked for clang-tidy contains the path to Catch2 as a system include directory yet the diagnostics is still printed for it. When trying to isolate it I have discovered that this does not actually work with clang-tidy:
clang-tidy src.cpp -- -Isystem/Path/to/header
It results in the header not being found at all. What I have learned somewhere (cannot find it now) was to make it actually two --extra-arg parameters of the clang-tidy instead:
clang-tidy --extra-arg=-Isystem --extra-arg=/Path/to/header src.cpp
This however does not work everywhere. On Windows I was able to make it work but on Linux it never worked in any form (together, separate, after --). How does one use the -isystem headers with clang-tidy on Linux? It is very confusing and inconsistent. Furthermore how to do it with CMake?
I have this:
cmake_minimum_required(VERSION 3.10)
set(CMAKE_CXX_CLANG_TIDY clang-tidy)
add_library(Catch2 STATIC Catch2/Catch2.cpp Catch2/include/Catch2/catch.hpp)
target_include_directories(Catch2 SYSTEM PUBLIC Catch2/include)
add_executable(SomeTest SomeTest/test.cpp)
target_link_libraries(Catch2)
The generated command line is rather convoluted (wrapping is mine for readability):
cmake
-E __run_co_compile
--tidy="clang-tidy-10;--extra-arg-before=--driver-mode=g++"
--source=../Sometest/test.cpp
--
/usr/bin/clang++-10
-isystem ../Catch2/include
-g
-std=gnu++17
-MD
-MT CMakeFiles/SomeTest.dir/projects/SomeTest/test.cpp.o
-MF CMakeFiles/SomeTest.dir/projects/SomeTest/FileTest.cpp.o.d
-o CMakeFiles/SomeTest.dir/projects/SomeTest/test.cpp.o
-c
../projects/SomeTest/test.cpp
In the output there are warnings from the Catch2 so the system in the include is just ignored seemingly. I have tried to force the --extra-arg via the CMAKE_CXX_CLANG_TIDY property:
set(CMAKE_CXX_CLANG_TIDY clang-tidy --extra-arg=-isystem --extra-arg=../Catch2/include)
but that does not seem to do the trick either.

I am following your repro as posted on LLVM bugtracker.
You are doing everything correctly: that is, marking Catch2 as system include with SYSTEM. clang-tidy is also behaving correctly: it only checks your source file test.cpp and doesn't fully check catch.hpp, only the macro expansion.
The problem is the outdated version of Catch2. hicpp-vararg warning has been silenced as of Catch2 2.12.2, so you need to update to at least that version. Moreover, apparently the core issue that hicpp-vararg reported upon has been fixed and this change is expected to be present in clang-tidy 11 release.

Related

LTO with LLVM and CMake

I am trying to apply Link Time Optimization with LLVM on a CMake Project, that creates a shared library. My question is pretty much the same as this one:
Switching between GCC and Clang/LLVM using CMake.
However, the answers do not seem to be applicable anymore, since llvm-ld is not present in the new versions. On the command line, I run the following commands to get LTO (Assuming there are only 2 .cpp files):
Compile to byte code:
clang++ -c FirstClass.cpp -O3 -flto -o FirstClass.bc
clang++ -c SecondClass.cpp -O3 -flto -o SecondClass.bc
Link byte code:
llvm-link FirstClass.bc SecondClass.bc -o unoptimized.bc
Optimize byte code:
opt -O3 unoptimized.bc -o optimized.bc
Convert byte code to shared object:
clang++ -shared optimized.bc -o libTest.so
Could somebody please tell me how to have CMake run the additional steps?
The correct way to use Clang and enable LTO is using the -flto flag to the clang command line both at compile and link time.
In addition, you will need to be working on a platform with a linker that either directly supports LTO (Apple's platforms generally) or that have an LLVM linker plugin (Linux using the Gold linker, but I think some have gotten the BFD linker to support the linker plugin as well). If you're using the linker plugin, you'll need to make sure your install of LLVM built and installed the plugin. If it did, Clang will automatically add the necessary linker command line options to use the plugin when linking with -flto, even for shared objects.
Also, The LLVM project is working on a new linker (LLD) which will support LTO out of the box on all the platforms it supports, but it is still pretty early days. Currently I know of folks testing out its LTO support on Windows and Linux, and it seems to be working well but still misses many features.
check_ipo_supported() resulted for me in "Policy CMP0069 is not set" error on CMake 3.9.1.
Per its help, CMake up to 3.8 only supported Intel compiler's LTO. It didn't work on XCode 9's clang for me either.
What worked, in the end:
cmake_policy(SET CMP0069 NEW)
include(CheckIPOSupported)
check_ipo_supported()
add_executable(Foobar SOURCES)
set_target_properties(Foobar PROPERTIES INTERPROCEDURAL_OPTIMIZATION TRUE)
Looks like add_executable() needs to be after cmake_policy(SET CMP0069 NEW).
LTO cache
target_link_libraries(Foobar "-Wl,-cache_path_lto,${PROJECT_BINARY_DIR}/lto.cache") did no harm.
Pick your command-line option depending on your linker.
More brutal option
According to #ChandlerCarruth's answer:
if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -flto")
target_link_libraries(Foobar -flto)
endif ()
Enabling (thin) lto on Cmake 3.9 and newer should be straightforward:
include(CheckIPOSupported)
check_ipo_supported()
set_target_properties(myProject PROPERTIES INTERPROCEDURAL_OPTIMIZATION TRUE)
Instead of set_target_properties per project, a single global setting of set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE) can be done.
In order to speed up recompiles, a cache for LTO can be set:
function(append value)
foreach(variable ${ARGN})
set(${variable} "${${variable}} ${value}" PARENT_SCOPE)
endforeach(variable)
endfunction()
append("-fuse-ld=gold -Wl,--no-threads,--plugin-opt,cache-dir=${PROJECT_BINARY_DIR}/lto.cache" CMAKE_EXE_LINKER_FLAGS CMAKE_SHARED_LINKER_FLAGS)
This forces gold as linker, in order to use the right command line options. It might require a symlink of /usr/lib/LLVMgold.so to /usr/lib/llvm-4.0/lib/LLVMgold.so.

Rcpp Library Won't Build (Can't find Compiler) on Ubuntu

I have a package that depends on Rcpp and uses two other libraries compiled from sub-directories in src/. The package builds fine on Mac OSX using a clang compiler. However, on an RStudio Ubuntu server, it fails to build. The build's first two steps (creating the static libraries in the sub directories to link in) work fine and I can see sensible build commands like the following taking place:
g++ -Wall -I../../inst/include/ --std=c++11 -lhts -L../htslib/ -lz -lm -c -o someLibFile.o someLibFile.cpp
However, in the very last step of the build process where it tries to build the Rcpp code and bind to the library, for some reason it appears to compleletey fail to put the compiler command in front (g++) and only outputs the second half of the command.
-o mypackage.so RcppExports.o temp.o -lhts -lpbbam -Lpbbam/ -L/htslib/ -Lpbbam/ -L/mnt/software/r/R/3.1.1/usr/lib/R/lib -lR
In contrast, on the Mac it builds just fine, appending clang++ and other flags in front of this final command:
clang++ -std=c++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o pbbamr.so LoadData.o RcppExports.o temp.o -lhts -lpbbam -Lpbbam/ -Lhtslib/ -Lpbbam/ -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
How do I make it use the g++ compiler on Ubuntu at this step? I have a custom Makevars file, but it is there just to build the dependencies in the sub-directory, so I don't know why that would cause any problems (since it works on Mac OSX).
More Information
The compiler seems to be found if I delete my Makevars file. However, the Makevars file I am using is essentially a direct copy of the example given in the R extensions guide with one addition to enable C++11:
CXX_STD = CXX11
.PHONY: all mylibs
all: $(SHLIB)
$(SHLIB): mylibs
mylibs:
(cd subdir; make)
With the line CXX_STD removed, it does stick a compiler in front of the command.
Briefly:
What is your R installation? You should probably run the binaries provided by Michael via CRAN; they are based on my Debian upload; I run these too on a bunch of machines
The reason is that R 'remembers' its compile-time settings via $RHOME/etc/Makefconf. This should just be CXX=g+=.
When you install r-base-dev (from Ubuntu or the newer version from CRAN) you also get the build-essential package as well as all common dependencies. With that things just work.
If however you are doing something special or local, well then you have to deal with your local changes. The basic Ubuntu setup is used by thousands of people and daily jobs--including eg Travis builds for countless GitHub repos.
This is caused by using an outdated/unusual R installation which has poor support for C++11. The best way to resolve his is to upgrade to a more recent version of R, or use a standard R install (sudo apt-get install r-base-dev). A poor work around is described below.
Problems Cause and Bad Work Around
When writing R extension that use C++11, one often sets CXX_STD = CXX11 in the Makevars file or list SystemRequirements: C++11 in the DESCRIPTION file. These will trigger R to use the compiler set by the following flags in the Makeconf file (located at file.path(R.home(), "etc/Makeconf")).
CXX1X
CXX1XFLAGS
CXX1XPICFLAGS
CXX1XSTD
Note that some of these may be set in this file, but not all of them might be there indicating a problem. In the event there is a problem with these settings or they are not set, R appears to use the empty string "" as the compiler/linker for the C++ code, leading to the problem shown above where no compiler argument is given.
If upgrading is not an option and you need to deploy on a known machine, one work around is to manually setup for C++11 by making a more idiosyncratic Makevars file. For example, you could:
Remove the CXX_STD=CXX11 line from the Makevars file.
Remove SystemRequirements: C++11 from the DESCRIPTION file.
Add --std=c++11 and any other requirements needed to PKG_CPPFLAGS, PKG_CFLAGS, PKG_CXXFLAGS or whatever variable is being used to compile your code, to manually set the needed flags (assuming the machine's compiler actually does support C++11).
The above solution is not particularly robust, but can be used as a work around in case the machine cannot be upgraded.
Thanks to #DirkEddelbuettel for not only writing Rcpp but being willing to support it on StackOverflow and help with issues like this.

Linking error when compiling Crypto++ for ARMHF

I'm trying to compile the crypto++ library to run for the armhf architecture. I'm following the method provided in this answer. I tweaked the setenv-embed.sh to match my system's configuration. The output of running . ./setenv-embed.sh is
CPP: /usr/bin/arm-linux-gnueabihf-cpp
CXX: /usr/bin/arm-linux-gnueabihf-g++
AR: /usr/bin/arm-linux-gnueabihf-ar
LD: /usr/bin/arm-linux-gnueabihf-ld
RANLIB: /usr/bin/arm-linux-gnueabihf-gcc-ranlib-4.8
ARM_EMBEDDED_TOOLCHAIN: /usr/bin
ARM_EMBEDDED_CXX_HEADERS: /usr/arm-linux-gnueabihf/include/c++/4.8.2
ARM_EMBEDDED_FLAGS: -march=armv7-a mfloat-abi=hard -mfpu=neon -I/usr/arm-linux-gnueabihf/include/c++/4.8.2 -I/usr/arm-linux-gnueabihf/include/c++/4.8.2/arm-linux-gnueabihf
ARM_EMBEDDED_SYSROOT: /usr/arm-linux-gnueabihf
which indicates that the correct compilers have been found. However, when I build the library using make I run into the following error
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/libc.so.6 inside /usr/arm-linux-gnueabihf
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/libc_nonshared.a inside /usr/arm-linux-gnueabihf
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 inside /usr/arm-linux-gnueabihf
But when I open the location /usr/arm-linux-gnueabihf/lib I can find all the three error files mentioned above ie libc.so.6, libc_nonshared.a and ld-linux-armhf.so.3
I'm trying to compile the library for Beaglebone, if that helps.
Update 1:
The results of running make -f GNUmakefile-cross system after doing a fresh git pull
hassan#hassan-Inspiron-7537:~/cryptopp-armhf$ make -f GNUmakefile-cross system
CXX: /usr/bin/arm-linux-gnueabihf-g++
CXXFLAGS: -DNDEBUG -g2 -Os -Wall -Wextra -DCRYPTOPP_DISABLE_ASM -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb -I/usr/arm-linux-gnueabihf/include/c++/4.8.2 -I/usr/arm-linux-gnueabihf/include/c++/4.8.2/arm-linux-gnueabihf --sysroot=/usr/arm-linux-gnueabihf -Wno-type-limits -Wno-unknown-pragmas
LDLIBS:
GCC_COMPILER: 1
CLANG_COMPILER: 0
INTEL_COMPILER: 0
UNALIGNED_ACCESS:
UNAME: Linux hassan-Inspiron-7537 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
MACHINE:
SYSTEM:
RELEASE:
make: Nothing to be done for `system'.
The problem is simple. It is in the --sysroot option. The value of this option is /usr/arm-linux-gnueabihf/ and it is used by the linker and the resulting library folder becomes
/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf/lib/
I removed the --sysroot option from line 68 in the file GNUmakefile-cross and everything compiled and linked OK.
However, I couldn't run the example on my BeagleBone Black because of mismatch of some shared libraries versions. But this wasn't a real problem for me, because in my application I link crypto++ statically, not dynamically.
Based on Crosswalking's research I think I can explain what is going on. I don't think I agree with the assessment "The problem is simple. It is in the --sysroot option" since the Crypto++ environment script and makefile are doing things as expected.
I think Crosswalking's answer could be how to work around it; but see open questions below. The following is from Crypto++ Issue 134: setenv-embedded.sh and GNUmakefile-cross:
I think this another distro problem, similar to g++-arm-linux-gnueabi
cannot compile a C++ program with
--sysroot.
It might be a Ubuntu problem or a Debian problem if it is coming from
upstream.
When cross-compiling, we expect the following (using ARMHF):
SYSROOT is /usr/arm-linux-gnueabihf
INCLUDEDIR is /usr/arm-linux-gnueabihf/include
LIBDIR is /usr/arm-linux-gnueabihf/lib
BINDIR is /usr/arm-linux-gnueabihf/bin
How LIBDIR morphed into into
/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf/lib/ (i.e.,
$SYSROOT/$SYSROOT/lib) is a mystery. But in all fairness, building
GCC is not a trivial task.
You should probably file a bug report with Debian or Ubuntu (or
whomever provides the toolchain).
The open question for me is, since $SYSROOT/lib is messed up, then is $SYSROOT/include messed up, too?
If the include directory is also messed up, then the cross compile is using the host's include files, and not the target include files. That will create hard to diagnose problems later.
If both $SYSROOT/include and $SYSROOT/lib are messed up, then its not enough to simply remove --sysroot. Effectively, this is what has to be done:
# Exported by setenv-embedded
export=ARM_EMBEDDED_SYSROOT=/usr/arm-linux-gnueabihf
# Used by the makefile
-I $ARM_EMBEDDED_SYSROOT/$ARM_EMBEDDED_SYSROOT/include
-L $ARM_EMBEDDED_SYSROOT/$ARM_EMBEDDED_SYSROOT/lib
Which means we should be able to do the following:
# Exported by setenv-embedded
export=ARM_EMBEDDED_SYSROOT=/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf
# Used by the makefile
--sysroot="$ARM_EMBEDDED_SYSROOT"
Finally, this looks a lot like Ubuntu's Bug 1375071: g++-arm-linux-gnueabi cannot compile a C++ program with --sysroot. The bug report specifically calls out ... the built-in paths use an extra "/usr/arm-linux-gnueabi".
We need the paths:
A) /usr/arm-linux-gnueabi/include/c++/4.7.3 B)
/usr/arm-linux-gnueabi/include/c++/4.7.3/arm-linux-gnueabi
But the built-in paths tries to use:
C) /usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3
D)
/usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3/arm-linux-gnueabi/sf
E)
/usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3/backward
Notice the built-in paths use an extra "/usr/arm-linux-gnueabi"

Error: ‘to_string’ is not a member of ‘std’ when compiling in command line with gcc 4.8.2, works in NetBeans?

I am doing tests for my code with GoogleTest (1.7.0). I must have the tests compiling in both NetBeans (8.0.2) and straight from command line.
NetBeans has GoogleTest integrated as suggested here: https://youtu.be/TS2CTf11k1U.
I have also build GoogleTest from the command line (in different place) with the instructions provided with the package.
The problematic line of code is (a and b are doubles):
std::string input = std::to_string(a) + " " + std::to_string(b);
which gives error: ‘to_string’ is not a member of ‘std’, when compiling with
g++ -isystem -std=c++0x ~/gtest-1.7.0/include -pthread ~/Task1/test_source.cpp libgtest.a -o test_module1_1
as instructed in GoogleTest documentation.
The strange thing here is that when using the "Test" from NetBeans (as seen on video), the tests compile and run correctly.
I am using Ubuntu 14.04 and I have already tried to update g++ (not to a higher version though), and I have also tried specifying the version with -std=c++11 instead of -std=c++0x.
In NetBeans there is this warning visible:
Library File /usr/include/c++/4.8/string
but there is an unresolved #include <stddef.h>
in included /usr/include/_G_config.h
I found out that this might be because multilib and I have also tried to fix that by checking that it is installed. And this doesn't really cause any warnings during compile so it might be just that the IDE is confused.
But conserning the real problem about compile errors:
Is there something wrong with the GoogleTest framework, the compiler or my code?
PS. Preferably don't suggest that I just don't use to_string as this system must be usable by students studying C++11 and really it shouldn't be that they can't use all the functionality that should be supported.
I think your problem is the order in which you're passing command line arguments to gcc
g++ -isystem -std=c++0x ~/gtest-1.7.0/include ...
^^^^^^^^^^^^^^^^^^^
The -isystem flag should be followed by a path that contains headers which you want gcc to treat as system headers. So the -std=c++0x flag is being incorrectly consumed by the -isystem flag. You probably want the command line to be
g++ -std=c++0x -isystem ~/gtest-1.7.0/include ...

why self built g++ compiler fails to compile my code

I wanted to use latest g++ compiler(4.9.1) on suse linux, but suse only supports an older g++ version. So, I took a latest source code from one of the gnu mirror sites and compiled it myself. Everything went fine. But when I tried to compile my test code using the built g++, the compilation fails with error,
"/root/home/include/c++/4.9.1/x86_64-unknown-linux-gnu/bits/os_defines.h:39:22: fatal error: features.h: No such file or directory".
I can find a "features.h" in "/root/home/include/c++/4.9.1/parallel", but I feel that it should be there in "/root/home/include/c++/4.9.1/" itself.
I copied "/root/home/include/c++/4.9.1/parallel/features.h" to "/root/home/include/c++/4.9.1/" just to see what happens. Now it complains with error "whcar.h" not found.
Have I missed something.
Here are the steps I followed to build g++.
1. /root/home/gcc_build/objdir# ../gcc-4.9.1/configure --prefix=/root/home/ --disable-multilib
2. /root/home/gcc_build/objdir# make -j16
3. /root/home/gcc_build/objdir# make install
4. /root/home/gcc_build/test# /root/home/bin/g++ --sysroot /root/home -m64 test.cpp
I resolved the issue by removing sysroot option and pointing c++ include and library path to my home directory. One thing I noticed was that the g++ source does not come with libc and c header files, and libc has to be installed seperately. But with sysroot option, g++ was trying to look all the header files in my home directory.
Below is the command I used to successfully compile the code.
/root/home/bin/g++ -I /root/home/include/c++/4.9.1 -L /root/home/lib64 -Wl,--rpath=/root/home/lib64 --std=c++0x -m64 test.cpp
Take a look at the GCC Directory Options. It is important to use the correct "specifier" as well (-isystem, -L, -B, -I etc)