My project uses SCons to manage the build process. I want to support multiple compilers, so I decided to use AddOption so the user can specify which compiler to use on the command line (with the default being whatever their current compiler is).
AddOption('--compiler', dest = 'compiler', type = 'string', action = 'store', default = DefaultEnvironment()['CXX'], help = 'Name of the compiler to use.')
I want to be able to have built-in compiler settings for various compilers (including things such as maximum warning levels for that particular compiler). This is what my first attempt at a solution currently looks like:
if is_compiler('g++'):
from build_scripts.gcc.std import cxx_std
from build_scripts.gcc.warnings import warnings, warnings_debug, warnings_optimized
from build_scripts.gcc.optimizations import optimizations, preprocessor_optimizations, linker_optimizations
elif is_compiler('clang++'):
from build_scripts.clang.std import cxx_std
from build_scripts.clang.warnings import warnings, warnings_debug, warnings_optimized
from build_scripts.clang.optimizations import optimizations, preprocessor_optimizations, linker_optimizations
However, I'm not sure what to make the is_compiler() function look like. My first thought was to directly compare the compiler name (such as 'clang++') against what the user passes in. However, this immediately failed when I tried to use scons --compiler=~/data/llvm-3.1-obj/Release+Asserts/bin/clang++.
So I thought I'd get a little smarter and use this function
cxx = GetOption('compiler')
def is_compiler (compiler):
return cxx[-len(compiler):] == compiler
This only looks at the end of the compiler string, so that it ignores directories. Unfortunately, 'clang++' ends in 'g++', so my compiler was seen to be g++ instead of clang++.
My next thought was to do a backward search and look for the first occurrence of a path separator ('\' or '/'), but then I realized that this won't work for people who have multiple compiler versions. Someone compiling with 'g++-4.7' will not register as being g++.
So, is there some simple way to determine which compiler was requested?
Currently, only g++ and clang++ are supported (and only their most recently released versions) due to their c++11 support, so a solution that only works for those two would be good enough for now. However, my ultimate goal is to support at least g++, clang++, icc, and msvc++ (once they support the required c++11 features), so more general solutions are preferred.
Compiler just are part of build process. Also you need linker tool and may be other additional programs. In Scons it's named - Tool. List of tools supported from box you can see in man page, search by statement: SCons supports the following tool specifications out of the box: ...
Tool set necessary scons environment variables, it's documented here.
Scons automatically detects compiler in OS and have some priority to choose one of them, of course autodetect will work properly if PATH variable set to necessary dirs. For example of you have msvc and mingw on windows, scons choose msvc tool. For force using tool use Tool('name')(env). For example:
env = Environment()
Tool('mingw')(env)
Now env force using mingw.
So, clang is one of tool which currently not supported from box by scons. You need to implement it, or set env vars such CC, CXX which using scons for generate build commands.
You could just simply use the Python os.path.basename() or os.path.split() functions, as specified here.
You could do what people suggested in the comments by splitting this question into 2 different issues, but I think it could be a good idea to be able to specify the path with the compiler, since you could have 2 versions of g++ installed, and if the user only specifies g++, they may not get the expected version.
There seems to be some confusion about what question is asked here.
For what I can see, this asks how to determine which compiler was chosen by default, so I'll answer that one.
From what I found out, the official way to check the compiler is to look at the construction variable TOOLS, which contains a list of all tools / programs that SCons decided / was told to use in the given construction environment.
env = Environment()
is_gcc = 'g++' in env['TOOLS']
is_clang = 'clangxx' in env['TOOLS']
TOOLS lists only the currently used tools even if SCons can find more of them.
E.g. if you have both GCC and Clang installed and SCons is able to find both, default TOOLS will still contain only GCC.
You can find the full list of predefined tools here.
Related
Having read the Meson site pages (which are generally high quality), I'm still unsure about the intended best practice to handle different options for different buildtypes.
So to specify a debug build:
meson [srcdir] --buildtype=debug
Or to specify a release build:
meson [srcdir] --buildtype=release
However, if I want to add b_sanitize=address (or other arbitrary complex set of arguments) only for debug builds and b_ndebug=true only for release builds, I would do:
meson [srcdir] --buildtype=debug -Db_sanitize=address ...
meson [srcdir] --buildtype=release -Db_ndebug=true ...
However, it's more of a pain to add a bunch of custom arguments on the command line, and to me it seems neater to put that in the meson.build file.
So I know I can set some built in options thusly:
project('myproject', ['cpp'],
default_options : ['cpp_std=c++14',
'b_ndebug=true'])
But they are unconditionally set.
So a condition would look something like this:
if get_option('buildtype').startswith('release')
add_project_arguments('-DNDEBUG', language : ['cpp'])
endif
Which is one way to do it, however, it would seem the b_ndebug=true way would be preferred to add_project_arguments('-DNDEBUG'), because it is portable.
How would the portable-style build options be conditionally set within the Meson script?
Additionally, b_sanitize=address is set without any test whether the compiler supports it. I would prefer for it to check first if it is supported (because the library might be missing, for example):
if meson.get_compiler('cpp').has_link_argument('-fsanitize=address')
add_project_arguments('-fsanitize=address', language : ['cpp'])
add_project_link_arguments('-fsanitize=address', language : ['cpp'])
endif
Is it possible to have the built-in portable-style build options (such as b_sanitize) have a check if they are supported?
I'm still unsure about the intended best practice to handle different options for different buildtypes
The intended best practice is to use meson configure to set/change the "buildtype" options as you need it. You don't have to do it "all at once and forever". But, of course, you can still have several distinct build trees (say, "debug" and "release") to speed up the process.
How would the portable-style build options be conditionally set within the Meson script?
Talking of b_ndebug, you can use the special value: ['b_ndebug=if-release'], which does exactly what you want. Also, you should take into account, that several GNU-style command-line arguments in meson are always portable, due to the internal compiler-specific substitutions. If I remember correctly, these include: -D, -I, -L and -l.
However, in general, changing "buildtype" options inside a script (except default_options, which are meant to be overwritten by meson setup/configure), is discouraged, and meson intentionally lacks set_option() function.
Is it possible to have the built-in portable-style build options (such as b_sanitize) have a check if they are supported?
AFAIK, no, except has_argument() you've used above. However, if some build option, like b_sanitize, is not supported by the underlying compiler, then it will be automatically set to void, so using it shouldn't break anything.
I want to build boost and other packages in buildroot with -std=c++11?
Is it possible to pass it globally, instead by patching program .mk files ?
There is no easy way to pass it globally, and for a good reason: some packages may not build with C++11, e.g. because they use new reserved words.
If you really want to risk it, however, you have three options:
Add -std=c++11 to BR2_TARGET_OPTIMIZATION (in the Toolchain menu). This will be included in the toolchain wrapper and therefore be used for each compile. Note that for C programs, this will give you "command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C" warnings. So packages that to -Werror will break.
Modify package/Makefile.in and add -std=c++11 to TARGET_CXXFLAGS. In this case, it's only passed to C++ compiles. However, TARGET_CXXFLAGS is just passed to the package build system and not all build systems honor it.
Modify toolchain/toolchain-wrapper.c to add this option when g++ is called. This doesn't have the disadvantages of the other two, but is more work to implement.
I work in a very regulated environment where we need to be able to produce identical binary input give the same source code every time be build out products. We currently use an ancient version of g++ that has been patched to not write anything like a date/time in the resulting binaries that would change from build to build, but I would like to update to g++ 4.7.2. Does anyone know of a patch, or have suggestions of what I need to look for to take two identical pieces of source code and produce identical binary outputs?
The Debian Reproducible builds project attempts to standardize Debian packages byte-by-byte, and has received a Linux Foundation grant in 2016.
While this may include more than compilation, you should have a look at it.
It also pointed me to this article, which adds the following points to what #Employed said:
put the source in a fixed folder (e.g. /tmp/build) to deal with __FILE__
for __DATE__, __TIME__, __TIMESTAMP__:
libfaketime : https://github.com/wolfcw/libfaketime
override those macros with -D
-Wdate-time or -Werror=date-time: warn or fail if either __TIME__, __DATE__ or __TIMESTAMP__ are is used. The Linux kernel 4.4 uses it by default.
use the D flag with ar, or use https://github.com/nh2/ar-timestamp-wiper/tree/master to wipe stamps
-fno-guess-branch-probability: older manual versions say it is a source of non-determinism, but not anymore. Not sure if this is covered by -frandom-seed or not.
Buildroot has a BR2_REPRODUCIBLE option which may give some ideas on the package level, but it is far from complete at this point.
Related threads:
https://superuser.com/questions/639351/does-recompiling-a-program-produce-a-bit-for-bit-identical-binary
https://www.quora.com/What-can-be-the-possible-reasons-for-the-object-code-of-an-unchanged-C-file-to-change-on-recompilation
We also depend on bit-identical rebuilds, and are using gcc-4.7.x.
Besides setting PWD=/proc/self/cwd and using -frandom-seed=<input-file-name>, there are a handful of patches, which can be found in svn://gcc.gnu.org/svn/gcc/branches/google/gcc-4_7 branch.
Use of the 'DATE' macro makes the build non-deterministic
I'm working on a C++ source analyzer project and it seems that clang is nice candidate for
the parsing work. The problem is that clang heavily depends on the infrastructure "llvm" project,
How do I configure it to get a clean front-end without any concrete machine oriented backend?
Just like LCC does, they provide a "null" backend for people who focus on parser parts.
Any suggestion is appreciated.
I recently did this on Windows.
Download the clang and llvm source from here.
Install cmake and Python (contrary to the docs, you do need Python just to build clang; at least, cmake gives up if it can't find a Python runtime).
You also need VS2008 or VS2010.
One thing that's not entirely obvious is the required directory structure:
projectRoot
build <- intermediate build files and DLLs, etc. will go here
llvm <- contents of llvm-3.0.src from llvm-3.0.tar go here
tools
clang <- contents of clang-3.0.src from clang-3.0.tar go here
And follow the windows build instructions from step 4 onwards. Don't attempt to use the cmake GUI, it's a horror; just use the commands given in the build instructions.
Once the build is complete (which takes a while) you'll have:
projectRoot
build
bin
Release <- libclang.dll will be here
lib
Release <- libclang.lib will be here
llvm
tools
clang
include
clang-c <- Index.h is here
Index.h defines the API to access information about your source code; it contains quite a bit of documentation about the APIs.
To get started using clang you need something like:
CXIndex index = clang_createIndex(1, 1);
// Support Microsoft extensions
char *args[] = {"-fms-extensions"};
CXTranslationUnit tu = clang_parseTranslationUnit(index, "mySource.c", args, ARRAY_SIZE(args), 0, 0, 0);
if (tu)
{
CXCursor cursor = clang_getTranslationUnitCursor(tu);
// Use the cursor functions to navigate through the AST
}
Unfortunately, you cannot get "pure" front-end without machine-specific details. C/C++ are inherently machine-tied languages. Think about preprocessor and built-in defines, the sizes of the builtin types, etc. Some of these can be abstracted out, but not e.g. preprocessor.
Using scons I can easily set my include paths:
env.Append( CPPPATH=['foo'] )
This passes the flag
-Ifoo
to gcc
However I'm trying to compile with a lot of warnings enabled.
In particular with
env.Append( CPPFLAGS=['-Werror', '-Wall', '-Wextra'] )
which dies horribly on certain boost includes ... I can fix this by adding the boost includes to the system include path rather than the include path as gcc treats system includes differently.
So what I need to get passed to gcc instead of -Ifoo is
-isystem foo
I guess I could do this with the CPPFLAGS variable, but was wondering if there was a better solution built into scons.
There is no built-in way to pass -isystem include paths in SCons, mainly because it is very compiler/platform specific.
Putting it in the CXXFLAGS will work, but note that this will hide the headers from SCons' dependency scanner, which only looks at CPPPATH.
This is probably OK if you don't expect those headers to ever change, but could cause weird issues if you use the build results cache and/or implicit dependency cache.
If you do
print env.Dump()
you'll see _CPPINCFLAGS, and you'll see that variable used in CCCOM (or _CCCOMCOM). _CPPINCFLAGS typically looks like this:
'$( ${_concat(INCPREFIX, CPPPATH, INCSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)'
From this you can probably see how you could add an "isystem" set of includes as well, like _CPPSYSTEMINCFLAGS or some such. Just define your own prefix, path var name (e.g. CPPSYSTEMPATH) and suffix and use the above idiom to concatenate the prefix. Then just append your _CPPSYSTEMINCFLAGS to CCCOM or _CCCOMCOM and off you go.
Of course this is system-specific but you can conditionally include your new variable in the compiler command line as and when you want.
According to the SCons release notes, "-isystem" is supported since version 2.3.4 for the environment's CCFLAGS.
So, you can, for example, do the following:
env.AppendUnique(CCFLAGS=('-isystem', '/your/path/to/boost'))
Still, you need to be sure that your compiler supports that option.
Expanding on the idea proposed by #LangerJan and #BenG... Here's a full cross-platform example (replace env['IS_WINDOWS'] with your windows platform checking)
from SCons.Util import is_List
def enable_extlib_headers(env, include_paths):
"""Enables C++ builders with current 'env' to include external headers
specified in the include_paths (list or string value).
Special treatment to avoid scanning these for changes and/or warnings.
This speeds up the C++-related build configuration.
"""
if not is_List(include_paths):
include_paths = [include_paths]
include_options = []
if env['IS_WINDOWS']:
# Simply go around SCons scanners and add compiler options directly
include_options = ['-I' + p for p in include_paths]
else:
# Tag these includes as system, to avoid scanning them for dependencies,
# and make compiler ignore any warnings
for p in include_paths:
include_options.append('-isystem')
include_options.append(p)
env.Append(CXXFLAGS = include_options)
Now, when configuring the use of external libraries, instead of
env.AppendUnique(CPPPATH=include_paths)
call
enable_extlib_headers(env, include_paths)
In my case this reduced the pruned dependency tree (as produced with --tree=prune) by 1000x on Linux and 3000x on Windows! It sped up the no-action build time (i.e. all targets up to date) by 5-7x
The pruned dependency tree before this change had 4 million includes from Boost. That's insane.