I have a waf build script and need to invoke a program which is not officially supported.
#file wscript
def configure(conf):
conf.env.LIB = ['c', 'd']
conf.env.INCLUDES = ['include']
conf.env.LIB_xml2 = ['xml2']
conf.env.INCLUDES_xml2 = ['/usr/include/libxml2']
def build(bld):
bld(rule="dstep ${SRC} -o ${TGT} ${LIB_ST:LIB} ${DINC_ST:INCLUDES}",
use="xml2",
source="header.h",
target="target.d",
)
This expands to dstep header.h -o target.d -lc -ld -I/usr/include/libxml2, so only the global LIB variable works, the use parameter seems to entirely ignored.
How can I make it respect the use parameter?
For waf to process the use keyword, you must add the use feature to your task generator. You also need to add a "compile" aware feature like c, d or cxx. Like this:
def build(bld):
bld(
rule="dstep ${SRC} -o ${TGT} ${LIB_ST:LIB} ${DINC_ST:INCLUDES}",
features = ["use", "c"],
use="xml2",
source="header.h",
target="target.d",
)
You can also program specifics for your dstep (You have to define a dstep feature and define the USELIB_VARS["dstep"])
Note: See ccroot.py in waf code.
Related
I need to attach to the SCons build in order to be notified when things happened during it: file compilation, files linkage, etc.
I know similar is possible for ANT builds via -listener option. Could you please tell how to do it for SCons builds?
When targets are built in SCons, you can associate a post action via the AddPostAction(target, action) function as documented here.
Here is a simple example with a python function action:
# Create yourAction here:
# can be a python function or external (shell) command line
def helloWorldAction(target = None, source = None, env = None):
'''
target: a Node object representing the target file
source: a Node object representing the source file
env: the construction environment used for building the target file
The target and source arguments may be lists of Node objects if there
is more than one target file or source file.
'''
print "PostAction for target: %s" % str(target)
# you can get a map of the source files like this:
# source_file_names = map(lambda x: str(x), source)
# compilation options, etc can be retrieved from the env
return 0
env = Environment()
progTarget = env.Program(target = "helloWorld", source = "helloWorld.cc")
env.AddPostAction(progTarget, helloWorldAction)
# Or create the action object like this:
# a = Action(helloWorldAction)
Then, each time helloWorld is built, the helloWorldAction python function will be executed afterwords.
Regarding doing this without modifying the given SConstruct, I dont see how that would be possible.
I have a running project made in qt . For building purpose I m using waf build tool. To get the same project up and running from waf I need to add
#include "file.moc"
at the end of some files to avoid undefined reference. But if these includes are not commented in qt I get can not find file errors. How do you make qt ignore certain file includes. I thought something like this should have done the trick
#ifndef Q_MOC_RUN
#include "file.moc"
#endif
Due to the limited information provided, all I can show is what waf can do:
You either can include the moc files or the unprocessed files.
Examples are included in the distributed source at https://code.google.com/p/waf/downloads/detail?name=waf-1.6.11.tar.bz2
Subdirectories:
playground/slow_qt/
demos/qt4/
For the sake of completness, simplified examples:
default includes
def options(opt):
opt.load('compiler_cxx qt4')
def configure(conf):
conf.load('compiler_cxx qt4')
conf.load('slow_qt4')
def build(bld):
bld(
features = 'qt4 cxx cxxprogram',
uselib = 'QTCORE QTGUI QTOPENGL QTSVG',
source = 'some.cpp files.cpp',
includes = '.',
target = 'dummy',
)
moc cpp
def options(opt):
opt.load('compiler_cxx qt4')
def configure(conf):
conf.load('compiler_cxx qt4')
def build(bld):
bld(
features = 'qt4 cxx cxxprogram',
uselib = 'QTCORE QTGUI QTOPENGL QTSVG',
source = 'some.cpp files.cpp',
target = 'dummy',
includes = '.')
)
Now I am studying an open source fluid simulation called pabalos. I have some problems building my own program that links against the library.
The library is built from source using scons.
The directory of the project is :
[fred#suck palabos-v1.1r0]$ls
codeblocks/ examples/ jlabos/ pythonic/ SConstruct utility/
COPYING externalLibraries/ lib/ scons/ src/
I will refer to this as the project root directory!
The project's official building documentation says:
The library Palabos makes use of an on-demand compilation process. The
code is compiled the first time it is used by an end-user application,
and then automatically re-used in future, until a new compilation is
needed due to a modification of the code or compilation options.
In the examples directory, there are some example code directories, such as :
[fred#suck palabos-v1.1r0]$ls examples/showCases/rectangularChannel3d/*
examples/showCases/rectangularChannel3d/Makefile
examples/showCases/rectangularChannel3d/rectangularChannel3D.cpp
The Makefile of the example is:
[fred#suck rectangularChannel3d]$cat Makefile
##########################################################################
## Makefile for the Palabos example program rectangularChannel3D.
##
## The present Makefile is a pure configuration file, in which
## you can select compilation options. Compilation dependencies
## are managed automatically through the Python library SConstruct.
##
## If you don't have Python, or if compilation doesn't work for other
## reasons, consult the Palabos user's guide for instructions on manual
## compilation.
##########################################################################
# USE: multiple arguments are separated by spaces.
# For example: projectFiles = file1.cpp file2.cpp
# optimFlags = -O -finline-functions
# Leading directory of the Palabos source code
palabosRoot = ../../..
# Name of source files in current directory to compile and link with Palabos
projectFiles = rectangularChannel3D.cpp
# Set optimization flags on/off
optimize = true
# Set debug mode and debug flags on/off
debug = false
# Set profiling flags on/off
profile = false
# Set MPI-parallel mode on/off (parallelism in cluster-like environment)
MPIparallel = true
# Set SMP-parallel mode on/off (shared-memory parallelism)
SMPparallel = false
# Decide whether to include calls to the POSIX API. On non-POSIX systems,
# including Windows, this flag must be false, unless a POSIX environment is
# emulated (such as with Cygwin).
usePOSIX = true
# Path to external libraries (other than Palabos)
libraryPaths =
# Path to inlude directories (other than Palabos)
includePaths =
# Dynamic and static libraries (other than Palabos)
libraries =
# Compiler to use without MPI parallelism
serialCXX = g++
# Compiler to use with MPI parallelism
parallelCXX = mpicxx
# General compiler flags (e.g. -Wall to turn on all warnings on g++)
compileFlags = -Wall -Wnon-virtual-dtor
# General linker flags (don't put library includes into this flag)
linkFlags =
# Compiler flags to use when optimization mode is on
optimFlags = -O3
# Compiler flags to use when debug mode is on
debugFlags = -g
# Compiler flags to use when profile mode is on
profileFlags = -pg
##########################################################################
# All code below this line is just about forwarding the options
# to SConstruct. It is recommended not to modify anything there.
##########################################################################
SCons = $(palabosRoot)/scons/scons.py -j 2 -f $(palabosRoot)/SConstruct
SConsArgs = palabosRoot=$(palabosRoot) \
projectFiles="$(projectFiles)" \
optimize=$(optimize) \
debug=$(debug) \
profile=$(profile) \
MPIparallel=$(MPIparallel) \
SMPparallel=$(SMPparallel) \
usePOSIX=$(usePOSIX) \
serialCXX=$(serialCXX) \
parallelCXX=$(parallelCXX) \
compileFlags="$(compileFlags)" \
linkFlags="$(linkFlags)" \
optimFlags="$(optimFlags)" \
debugFlags="$(debugFlags)" \
profileFlags="$(profileFlags)" \
libraryPaths="$(libraryPaths)" \
includePaths="$(includePaths)" \
libraries="$(libraries)"
compile:
python $(SCons) $(SConsArgs)
clean:
python $(SCons) -c $(SConsArgs)
/bin/rm -vf `find $(palabosRoot) -name '*~'`
I know this makefile will call scons, and SConstruct file is in the project root dir as I have shown.
The SContstruct file is :
[fred#suck palabos-v1.1r0]$cat SConstruct
###########################################################
# Configuration file for the compilation of Palabos code,
# using the SConstruct library.
# IT IS NOT RECOMMENDED TO MODIFY THIS FILE.
# Compilation should be personalized by adjusting the
# Makefile in the directory of the main source files.
# See Palabos examples for sample Makefiles.
###########################################################
import os
import sys
import glob
argdict = dict(ARGLIST)
# Read input parameters
palabosRoot = argdict['palabosRoot']
projectFiles = Split(argdict['projectFiles'])
optimize = argdict['optimize'].lower() == 'true'
debug = argdict['debug'].lower() == 'true'
profile = argdict['profile'].lower() == 'true'
MPIparallel = argdict['MPIparallel'].lower() == 'true'
SMPparallel = argdict['SMPparallel'].lower() == 'true'
usePOSIX = argdict['usePOSIX'].lower() == 'true'
serialCXX = argdict['serialCXX']
parallelCXX = argdict['parallelCXX']
compileFlags = Split(argdict['compileFlags'])
linkFlags = Split(argdict['linkFlags'])
optimFlags = Split(argdict['optimFlags'])
debugFlags = Split(argdict['debugFlags'])
profileFlags = Split(argdict['profileFlags'])
libraryPaths = Split(argdict['libraryPaths'])
includePaths = Split(argdict['includePaths'])
libraries = Split(argdict['libraries'])
# Read the optional input parameters
try:
dynamicLibrary = argdict['dynamicLibrary'].lower() == 'true'
except:
dynamicLibrary = False
try:
srcPaths = Split(argdict['srcPaths'])
except:
srcPaths = []
flags = compileFlags
allPaths = [palabosRoot+'/src'] + [palabosRoot+'/externalLibraries'] + includePaths
if optimize:
flags.append(optimFlags)
if debug:
flags.append(debugFlags)
flags.append('-DPLB_DEBUG')
if profile:
flags.append(profileFlags)
linkFlags.append(profileFlags)
if MPIparallel:
compiler = parallelCXX
flags.append('-DPLB_MPI_PARALLEL')
else:
compiler = serialCXX
if SMPparallel:
flags.append('-DPLB_SMP_PARALLEL')
if usePOSIX:
flags.append('-DPLB_USE_POSIX')
env = Environment ( ENV = os.environ,
CXX = compiler,
CXXFLAGS = flags,
LINKFLAGS = linkFlags,
CPPPATH = allPaths
)
if dynamicLibrary:
LibraryGen = env.SharedLibrary
else:
LibraryGen = env.Library
sourceFiles = []
for srcDir in glob.glob(palabosRoot+'/src/*'):
sourceFiles.extend(glob.glob(srcDir+'/*.cpp'))
for srcDir in srcPaths:
sourceFiles.extend(glob.glob(srcDir+'/*.cpp'))
sourceFiles.extend(glob.glob(palabosRoot+'/externalLibraries/tinyxml/*.cpp'));
if MPIparallel:
palabos_library = LibraryGen( target = palabosRoot+'/lib/plb_mpi',
source = sourceFiles )
else:
palabos_library = LibraryGen( target = palabosRoot+'/lib/plb',
source = sourceFiles )
local_objects = env.Object(source = projectFiles)
all_objects = local_objects + palabos_library
env.Program(all_objects, LIBS=libraries, LIBPATH=libraryPaths)
My problem is:
When I changed the source file rectangularChannel3D.cpp in the example dir,
and run make, the palabos library should not be rebuilt since I didn't change
the library project's source file (in the 'src' dir of the root dir) at all. But
actually the lib file "libplb.a" had been rebuilt!! So why?
I agree with Brady. Try contacting the project you're trying to build.
Alternatively, if you get no help from them, and/or really want to fix this yourself, SCons has a flag --debug=explain which will tell you why any object is being build/rebuilt.
I am trying to compile a NodeJS native module, using two files: 1 .c file and 1 .cpp file. Here's what my 'wscript' file looks like:
def set_options(opt):
opt.tool_options("compiler_cxx")
def configure(conf):
conf.check_tool("compiler_cxx")
conf.check_tool("node_addon")
def build(bld):
obj = bld.new_task_gen("cxx")
obj.cxxflags = ["-g", "-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE", "-Wall", "-x", "objective-c"]
obj.source = "c-file.c"
obj = bld.new_task_gen("cxx", "shlib", "node_addon")
obj.cxxflags = ["-g", "-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE", "-Wall"]
obj.target = "binding"
obj.source = "cpp-file.cc"
This builds me a binding.node file which I can then partially use in Node, but as soon as I call the function that is located in the C file (the one compiled first in the wscript above), Node crashes with something like:
dyld: lazy symbol binding failed: Symbol not found: __Z9getSomethingv
Referenced from: /Users/nrajlich/test/build/default/binding.node
Expected in: flat namespace
This leads me to believe that the first file isn't being included in the linking phase, but I'm just not sure how I'm supposed to add it. Any ideas? Thanks in advance!
I don't think what I was originally trying to do is possible with waf.
In this case, I was trying to compile C++ and Obj-C code together and use them together. Originally I had the Obj-C code in the .c files and the C++ code in the .cc files, and was trying to pass different flags for the different file types.
Since then, I've learned that Obj-C and C++ may be combined in the same file! And I just needed to add an -ObjC++ flag to g++. This was easily done in waf, and also allowed to to consolidate all the needed stuff into a single file. Everything works great from there.
So I was able to solve what I originally was trying to do, however the original question I asked I don't believe is possible to do with waf. Cheers!
def set_options(opt):
opt.tool_options("compiler_cxx")
def configure(conf):
conf.check_tool("compiler_cxx")
conf.check_tool("node_addon")
def build(bld):
obj = bld.new_task_gen("cxx", "shlib", "node_addon")
obj.cxxflags = ["-g", "-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE", "-Wall"]
obj.target = "binding"
obj.source = """
cpp-file.cc
c-file.c
"""
That's how you should do this.
I'm trying to build an application I've downloaded which uses the SCONS "make replacement" and the Fast Light Tool Kit Gui.
The SConstruct code to detect the presence of fltk is:
guienv = Environment(CPPFLAGS = '')
guiconf = Configure(guienv)
if not guiconf.CheckLibWithHeader('lo', 'lo/lo.h','c'):
print 'Did not find liblo for OSC, exiting!'
Exit(1)
if not guiconf.CheckLibWithHeader('fltk', 'FL/Fl.H','c++'):
print 'Did not find FLTK for the gui, exiting!'
Exit(1)
Unfortunately, on my (Gentoo Linux) system, and many others (Linux distributions) this can be quite troublesome if the package manager allows the simultaneous install of FLTK-1 and FLTK-2.
I have attempted to modify the SConstruct file to use fltk-config --cflags and fltk-config --ldflags (or fltk-config --libs might be better than ldflags) by adding them like so:
guienv.Append(CPPPATH = os.popen('fltk-config --cflags').read())
guienv.Append(LIBPATH = os.popen('fltk-config --ldflags').read())
But this causes the test for liblo to fail! Looking in config.log shows how it failed:
scons: Configure: Checking for C library lo...
gcc -o .sconf_temp/conftest_4.o -c "-I/usr/include/fltk-1.1 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_THREAD_SAFE -D_REENTRANT"
gcc: no input files
scons: Configure: no
How should this really be done?
And to complete my answer, how do I remove the quotes from the result of os.popen( 'command').read()?
EDIT The real question here is why does appending the output of fltk-config cause gcc to not receive the filename argument it is supposed to compile?
There are 2 similar ways to do this:
1)
conf = Configure(env)
status, _ = conf.TryAction("fltk-config --cflags")
if status:
env.ParseConfig("fltk-config --cflags")
else:
print "Failed fltk"
2)
try:
env.ParseConfig("fltk-config --cflags")
except (OSError):
print 'failed to run fltk-config you sure fltk is installed !?'
sys.exit(1)
This is quite a complex problem with no quick answer
I have referred to the instructions for using pkg-config with scons at http://www.scons.org/wiki/UsingPkgConfig. The following question is also helpful
Test if executable exists in Python?.
But we need to go a little bit further with these.
So after much investigation I discovered os.popen('command').read() does not trim the trailing newline '\n' which is what caused the truncation of the arguments sent to GCC.
We can use str.rstrip() to remove the trailing '\n'.
Secondly, as config.log shows, the arguments which fltk-config provides, SCONS wraps up in double quotes before giving them to GCC. I'm not exactly sure of the specifics but this is because the output of fltk-config (via os.popen) contains space characters.
We can use something like strarray = str.split(" ", str.count(" ")) to split the output into substrings where the space characters occur.
It is also worth noting that we were attempting to append the fltk-config --ldflags to the wrong variable within the GUI environment, they should have been added to LINKFLAGS.
Unfortunately this is only half way to the solution.
What we need to do is:
Find the full path of an executable on the system
Pass arguments to an executable and capture its output
Convert the output into a suitable format to append to the CPPFLAGS and LINKFLAGS.
So I have defined some functions to help...
1) Find full path of executable on system:
( see: Test if executable exists in Python? )
def ExecutablePath(program):
def is_exe(fpath):
return os.path.exists(fpath) and os.access(fpath, os.X_OK)
fpath, fname = os.path.split(program)
if fpath:
if is_exe(program):
return program
else:
for path in os.environ["PATH"].split(os.pathsep):
exe_file = os.path.join(path, program)
if is_exe(exe_file):
return exe_file
return None
1b) We also need to test for executable existence:
def CheckForExecutable(context, program):
context.Message( 'Checking for program %s...' %program )
if ExecutablePath(program):
context.Result('yes')
return program
context.Result('no')
2) Pass arguments to executable and place the output into an array:
def ExecutableOutputAsArray(program, args):
pth = ExecutablePath(program)
pargs = shlex.split('%s %s' %(pth, args))
progout = subprocess.Popen( pargs , stdout=subprocess.PIPE).communicate()[0]
flags = progout.rstrip()
return flags.split(' ', flags.count(" "))
Some usage:
guienv.Append(CPPFLAGS = ExecutableOutputAsArray('fltk-config', '--cflags') )
guienv.Append(LINKFLAGS = ExecutableOutputAsArray('fltk-config', '--ldflags') )
guienv.Append(LINKFLAGS = ExecutableOutputAsArray('pkg-config', '--libs liblo') )