All C++ compilers that support vectorization allow some report (*) being emitted to verify if a loop was vectorized, each with their own compilation flag and format in the vectorization report.
I can enable the corresponding flag and visually inspect the report to check if a loop that I expect to be auto-vectorized was indeed auto-vectorized.
I would like to incorporate in my CMake build a step that checks this automatically and fails the build if it didn't auto-vectorize.
How can I do this using CMake?
Is there anyone that has solved this problem already, somehow?
Thanks in advance
(*)
MSVC https://learn.microsoft.com/en-us/cpp/build/reference/qvec-report-auto-vectorizer-reporting-level?view=msvc-160
gcc https://gcc.gnu.org/onlinedocs/gcc/Developer-Options.html
clang https://llvm.org/docs/Vectorizers.html#diagnostics
I would like to incorporate in my CMake build a step that checks this automatically and fails the build if it didn't auto-vectorize.
How can I do this in CMake?
Applying dynamic programming in steps:
Create an algorithm that can detect if the program was "auto-vectorized".
You can use some output from the compiler generated with special options, why not.
You could also dissasemble the code and find "the loop" and check for some instructions or syntax.
Then write a portable program that does implement that algorithm, preferably in some portable language.
Add a custom target to cmake configuration to run the check that could look like the following:
add_executable(final_exe sources.c...)
add_exececutable(check_if_vectorized sources.c...) # if compiled, choose your own language
add_custom_target(check_if_final_exe_is_vectorized
COMMENT "Check if final_exe was vectorized"
COMMAND $<TARGET_FILE:check_if_vectorized> $<TARGET_FILE:final_exe>
DEPENDS $<TARGET_FILE:check_if_vectorized> $<TARGET_FILE:final_exe>
)
You could also add the check with add_test and have it run like a test.
Is there anything that works out of the box?
No.
Related
I am wanting to measure the time it takes for my C++ video processing program to process a video. I am using CLion to write the program and have Cmake set up to compile and automatically run the program with a test video. However, in order to find execution time I have been using the following command in the MacOS terminal:
% time ./main ../Media/test_video.mp4
Is there a way for me to configure Cmake to automatically include time in the execution of ./main to streamline my process further?
So far I've tried using set(CMAKE_ARGS time "test_video.mp4") and some command line argument functions but they don't seem to be acting in the way that I'm looking for.
It is possible to use add_custom_target to do what you want. I'll not consider this option further as it seems abusing the build system for something it wasn't designed to do. Yet it may have an advantage over using CLion configuration: it would be available to be used outside of CLion. That advantage seems minor: why not run the desired command directly in those contexts?
The first CLion method would be to define an external tool which run time on the current build target. In File|Settings...|Tools|External Tools define a new tool with /bin/time as program, $CMakeCurrentProductFile$ $Prompt$ as arguments. When choosing that tools (in Tools|External Tools) it will now prompt you for the argument and then run /bin/time on the current target with the provided arguments. Advantage: you don't have to define the tool once, it will be available in every project. Inconvenients: the external tools are available in every project, thus it doesn't make sense to be more specific than $Prompt$ for the arguments and the working directory; it isn't possible to specify environment variables, it isn't possible to enforce the need of a build before running the command.
The second CLion method would to be define a Run/Debug Configuration. Again use /bin/time as program (chose "Custom Executable"), specify $CMakeCurrentProductFile$ as first argument (here it makes sense to provide the other arguments as desired, but note that $Prompt$ is still a valid choice if needed). Advantages: it makes sense to be as specific as needed; you have all the feature of configurations (environment variables, input redirections, specifying actions to be executed before the execution). Inconvenient: it doesn't work with other CLion features which assume that the program is the target such as the debugger, the profiler, ... so you may have to duplicate configurations to get them.
Note that the methods aren't exclusive: you can define an external tools and then add configurations for the case where it is more convenient.
I have written a program prg which can be run using bash with some subcommands like so
$ prg subcommand1
Output of subcomman1
$ prg subcommand2
Output of subcomman2
The program is written in C++11 and compiled with CMake (3.10+).
Now, I'd like to add another optional command, say optional_subcommand.
Optional - in a sense that a user can decide whether to compile program with it when configuring Cake.
To clarify, when the code is not compiled withoptional_subcommand support, the output should be something along this lines
$ prg optional_subcommand
Option not supported...
The optional_subcommand requires external library to be installed. Hence find_package(SomeLib REQUIRED) must be invoked at some stage by CMake. The code for optional_subcommand requires this library to compile.
All I can think of is writing some dummy code for optional_subcommand which is then replaced on request by CMake. I think this can be achieved be asking CMake to overwrite, say optional_subcommand.cpp with either dummy or not.
Is proposed solution sensible or perhaps there is a better way to achieve this?
CMake can define a preprocessor symbol and add extra files to your target (below, prg) if needed:
if(SomeLib_FOUND)
target_compile_definitions(prg PUBLIC HAVE_SOMELIB)
target_sources(prg PRIVATE src/optional_subcommand.cpp)
endif()
You can then wrap the command dispatching code for optional_subcommand with #ifdef HAVE_SOMELIB.
Having read the Meson site pages (which are generally high quality), I'm still unsure about the intended best practice to handle different options for different buildtypes.
So to specify a debug build:
meson [srcdir] --buildtype=debug
Or to specify a release build:
meson [srcdir] --buildtype=release
However, if I want to add b_sanitize=address (or other arbitrary complex set of arguments) only for debug builds and b_ndebug=true only for release builds, I would do:
meson [srcdir] --buildtype=debug -Db_sanitize=address ...
meson [srcdir] --buildtype=release -Db_ndebug=true ...
However, it's more of a pain to add a bunch of custom arguments on the command line, and to me it seems neater to put that in the meson.build file.
So I know I can set some built in options thusly:
project('myproject', ['cpp'],
default_options : ['cpp_std=c++14',
'b_ndebug=true'])
But they are unconditionally set.
So a condition would look something like this:
if get_option('buildtype').startswith('release')
add_project_arguments('-DNDEBUG', language : ['cpp'])
endif
Which is one way to do it, however, it would seem the b_ndebug=true way would be preferred to add_project_arguments('-DNDEBUG'), because it is portable.
How would the portable-style build options be conditionally set within the Meson script?
Additionally, b_sanitize=address is set without any test whether the compiler supports it. I would prefer for it to check first if it is supported (because the library might be missing, for example):
if meson.get_compiler('cpp').has_link_argument('-fsanitize=address')
add_project_arguments('-fsanitize=address', language : ['cpp'])
add_project_link_arguments('-fsanitize=address', language : ['cpp'])
endif
Is it possible to have the built-in portable-style build options (such as b_sanitize) have a check if they are supported?
I'm still unsure about the intended best practice to handle different options for different buildtypes
The intended best practice is to use meson configure to set/change the "buildtype" options as you need it. You don't have to do it "all at once and forever". But, of course, you can still have several distinct build trees (say, "debug" and "release") to speed up the process.
How would the portable-style build options be conditionally set within the Meson script?
Talking of b_ndebug, you can use the special value: ['b_ndebug=if-release'], which does exactly what you want. Also, you should take into account, that several GNU-style command-line arguments in meson are always portable, due to the internal compiler-specific substitutions. If I remember correctly, these include: -D, -I, -L and -l.
However, in general, changing "buildtype" options inside a script (except default_options, which are meant to be overwritten by meson setup/configure), is discouraged, and meson intentionally lacks set_option() function.
Is it possible to have the built-in portable-style build options (such as b_sanitize) have a check if they are supported?
AFAIK, no, except has_argument() you've used above. However, if some build option, like b_sanitize, is not supported by the underlying compiler, then it will be automatically set to void, so using it shouldn't break anything.
I work in a very regulated environment where we need to be able to produce identical binary input give the same source code every time be build out products. We currently use an ancient version of g++ that has been patched to not write anything like a date/time in the resulting binaries that would change from build to build, but I would like to update to g++ 4.7.2. Does anyone know of a patch, or have suggestions of what I need to look for to take two identical pieces of source code and produce identical binary outputs?
The Debian Reproducible builds project attempts to standardize Debian packages byte-by-byte, and has received a Linux Foundation grant in 2016.
While this may include more than compilation, you should have a look at it.
It also pointed me to this article, which adds the following points to what #Employed said:
put the source in a fixed folder (e.g. /tmp/build) to deal with __FILE__
for __DATE__, __TIME__, __TIMESTAMP__:
libfaketime : https://github.com/wolfcw/libfaketime
override those macros with -D
-Wdate-time or -Werror=date-time: warn or fail if either __TIME__, __DATE__ or __TIMESTAMP__ are is used. The Linux kernel 4.4 uses it by default.
use the D flag with ar, or use https://github.com/nh2/ar-timestamp-wiper/tree/master to wipe stamps
-fno-guess-branch-probability: older manual versions say it is a source of non-determinism, but not anymore. Not sure if this is covered by -frandom-seed or not.
Buildroot has a BR2_REPRODUCIBLE option which may give some ideas on the package level, but it is far from complete at this point.
Related threads:
https://superuser.com/questions/639351/does-recompiling-a-program-produce-a-bit-for-bit-identical-binary
https://www.quora.com/What-can-be-the-possible-reasons-for-the-object-code-of-an-unchanged-C-file-to-change-on-recompilation
We also depend on bit-identical rebuilds, and are using gcc-4.7.x.
Besides setting PWD=/proc/self/cwd and using -frandom-seed=<input-file-name>, there are a handful of patches, which can be found in svn://gcc.gnu.org/svn/gcc/branches/google/gcc-4_7 branch.
Use of the 'DATE' macro makes the build non-deterministic
My project uses SCons to manage the build process. I want to support multiple compilers, so I decided to use AddOption so the user can specify which compiler to use on the command line (with the default being whatever their current compiler is).
AddOption('--compiler', dest = 'compiler', type = 'string', action = 'store', default = DefaultEnvironment()['CXX'], help = 'Name of the compiler to use.')
I want to be able to have built-in compiler settings for various compilers (including things such as maximum warning levels for that particular compiler). This is what my first attempt at a solution currently looks like:
if is_compiler('g++'):
from build_scripts.gcc.std import cxx_std
from build_scripts.gcc.warnings import warnings, warnings_debug, warnings_optimized
from build_scripts.gcc.optimizations import optimizations, preprocessor_optimizations, linker_optimizations
elif is_compiler('clang++'):
from build_scripts.clang.std import cxx_std
from build_scripts.clang.warnings import warnings, warnings_debug, warnings_optimized
from build_scripts.clang.optimizations import optimizations, preprocessor_optimizations, linker_optimizations
However, I'm not sure what to make the is_compiler() function look like. My first thought was to directly compare the compiler name (such as 'clang++') against what the user passes in. However, this immediately failed when I tried to use scons --compiler=~/data/llvm-3.1-obj/Release+Asserts/bin/clang++.
So I thought I'd get a little smarter and use this function
cxx = GetOption('compiler')
def is_compiler (compiler):
return cxx[-len(compiler):] == compiler
This only looks at the end of the compiler string, so that it ignores directories. Unfortunately, 'clang++' ends in 'g++', so my compiler was seen to be g++ instead of clang++.
My next thought was to do a backward search and look for the first occurrence of a path separator ('\' or '/'), but then I realized that this won't work for people who have multiple compiler versions. Someone compiling with 'g++-4.7' will not register as being g++.
So, is there some simple way to determine which compiler was requested?
Currently, only g++ and clang++ are supported (and only their most recently released versions) due to their c++11 support, so a solution that only works for those two would be good enough for now. However, my ultimate goal is to support at least g++, clang++, icc, and msvc++ (once they support the required c++11 features), so more general solutions are preferred.
Compiler just are part of build process. Also you need linker tool and may be other additional programs. In Scons it's named - Tool. List of tools supported from box you can see in man page, search by statement: SCons supports the following tool specifications out of the box: ...
Tool set necessary scons environment variables, it's documented here.
Scons automatically detects compiler in OS and have some priority to choose one of them, of course autodetect will work properly if PATH variable set to necessary dirs. For example of you have msvc and mingw on windows, scons choose msvc tool. For force using tool use Tool('name')(env). For example:
env = Environment()
Tool('mingw')(env)
Now env force using mingw.
So, clang is one of tool which currently not supported from box by scons. You need to implement it, or set env vars such CC, CXX which using scons for generate build commands.
You could just simply use the Python os.path.basename() or os.path.split() functions, as specified here.
You could do what people suggested in the comments by splitting this question into 2 different issues, but I think it could be a good idea to be able to specify the path with the compiler, since you could have 2 versions of g++ installed, and if the user only specifies g++, they may not get the expected version.
There seems to be some confusion about what question is asked here.
For what I can see, this asks how to determine which compiler was chosen by default, so I'll answer that one.
From what I found out, the official way to check the compiler is to look at the construction variable TOOLS, which contains a list of all tools / programs that SCons decided / was told to use in the given construction environment.
env = Environment()
is_gcc = 'g++' in env['TOOLS']
is_clang = 'clangxx' in env['TOOLS']
TOOLS lists only the currently used tools even if SCons can find more of them.
E.g. if you have both GCC and Clang installed and SCons is able to find both, default TOOLS will still contain only GCC.
You can find the full list of predefined tools here.