I need a chain of file processing in my build-process. Workflow can be easily specified and built around a filename, only extension changes, like file.a -> file.b -> file.c. So, it's clearly a case for Make's declarative syntax. But as I see, for CMake this will look like an explicit add_custom_command for each file for each step of processing.
So, the question is if CMake supports substitutions like % from Make, so that only general rules for each step of processing would be required.
I imagine this like:
add_custom_command(OUTPUT %.b
COMMAND convert %.a > %.b
DEPENDS %.a)
add_custom_command(OUTPUT %.c
COMMAND convert %.b > %.c
DEPENDS %.b)
No, CMake does not have any support for patterns. People typically use macros as a workaround.
People don't use CMake because of the elegant syntax and its expressiveness.
You could use macros.
A simple example:
MACRO( TEST )
MESSAGE ( "HELLO ${ARGV}" )
ENDMACRO( TEST )
TEST("WORLD")
Related
I remember doing this in my advanced C++ class, but it's been a couple years and I forgot.
I want to make a sort of simple re-usable makefile where I just type "make programName" and it compiles programName.cpp into programName.exe.
I've looked for this for a couple hours, but haven't found what I need. I know this will get marked as duplicate, but at least point me to what I need.
Make has a built-in variable called $(MAKECMDGOALS), which contains all targets specified as command line parameters.
You can use it to generate recipes, like so:
# Stop if 0 or 2+ targets were specified.
$(if $(filter-out 1,$(words $(MAKECMDGOALS))),$(error Expected one target))
name := $(MAKECMDGOALS)
$(name).exe: $(name).cpp
g++ $^ -o $#
Adding a way to customize compiler executable and flags is left as an exercise to the reader. As well as conditionally removing .exe from the target name when building on Linux.
Why do you want your output to be suffixed with .exe? That's a Windows convention that is not followed on POSIX systems like Linux.
If you don't want that you don't even need a makefile at all. You can type make programName with no makefile and if you have a file programName.c, it will be built into an executable programName via make's built-in rules.
I've come across a build system that uses scons. Not being familiar at all with scons itself being a rather sophisticated framework I get very frustrated not being able to debug build issues.
I want scons to printout the fully expanded command line being invoke ( as you see with most build systems) I found out you could use the --debug=presub option but ( at least on OSX ) it is useless since it prints the value of unexpanded variables
for example:
Building build/obj/ios-uni-rel-sta-clang/common/libs/boost/libs/date_time/src/gregorian/date_generators.i386.o with action:
$SHCXX -o $TARGET -c $SHCXXFLAGS $SHCCFLAGS $_CCCOMCOM $SOURCES
There is also a VERBOSE=1 parameter you can supply on the scons command line but ( at least with the 2.3.4 ) version I got, it doesn't seem to be verbose much anything.
I'm not saying Scons is bad, but it is become a bit taxing and expansive to maintain :(
Anyone familiar with Scons? What module, where is the actual command gets invoked? I just want to add a few prints ...
Alternatively, how can you setup PyDev or PyCharm to hook up using the scons --debug=pdb? Did anyone this?
Somewhere in a SConstruct, SConscript, or some python module loaded by either (could be in site_scons under top dir) someone is changing the *COMSTR env variables.
It will look something like:
env['SHCXXCOMSTR'] = "Building $TARGET"
or:
for k in env.keys():
if k.endswith('COMSTR'):
env[k] = "Building $TARGET"
You'll want to comment out those lines.
The default SCons behavior is to show the command lines.
http://scons.org/doc/production/HTML/scons-man.html#cv-SHCXXCOMSTR
SHOWBUILD=1
For example:
$>scons SHOWBUILD=1
This worked for me
I'm using a tool chain where I do not know the names of all of the intermediate files.
E.g. I know that I start out with a foo.s, and go through several steps to get a foo.XXXX.sym and a foo.XXXX.hex, buried way down deep. And then running other tools on foo.XXXX.hex and foo.XXXX.sym, I eventually end up with something like final.results.
But, the trouble is that I don't know what the XXXX is. It is derived from some other parameters, but may be significantly transformed away from them.
Now, after running the tool/steps that generate foo.XXXX.{sym,hex}, I now typically scan the overall result directory looking for foo.*.{sym,hex}. I.e. I have code that can recognize the intermediate outputs, I just don't know exactly what the names will be.
I typically use make or scons - actually, I prefer scons, but my team highly prefers make. I'm open to other build tools.
What I want to do is be able to say (1) "make final.results", or "scons final.results", (2) and have it scan over the partial tree; (3) figure out that, while it does not know the full path, it definitely knows that it has to run the first step, (4) after that first step, look for and find the foo.XXX.* files; (5) and plug those into the dependency tree.
I.e. I want to finish building the dependency tree after the build has already started.
A friend got frustrated enough with scons' limitations in this area that he wrote his own build tool. Unfortunately it is proprietary.
I guess that I can create a first build graph, say in make with many .PHONY targets, and then after I get through the first step, generate a new makefile with the new names, and have the first make invoke the newly generated second makefile. Seems clumsy. Is there any more elegant way?
GNU make has an "auto-rexec" feature that you might be able to make use of. See How Makefiles Are Remade
After make finishes reading all the makefiles (both the ones found automatically and/or on the command line, as well as all included makefiles), it will try to rebuild all its makefiles (using the rules it knows about). If any of those makefiles are automatically rebuilt, then make will re-exec itself so it can re-read the newest versions of the makefiles/included files, and starts over (including re-trying to build all the makefiles).
It seems to me that you should be able to do something with this. You can write in your main makefile and "-include foo.sym.mk" for example, and then have a rule that builds "foo.sym.mk" by invoking the tool on foo.s, then running your "recognized the next step" code and generate a "foo.sym.mk" file which defines a rule for the intermediate output that got created. Something like (due to lack of specificity in your question I can't give true examples you understand):
SRCS = foo.s bar.s baz.s
-include $(patsubst %.s,%.sym.mk,$(SRCS))
%.sym.mk: %.s
<compile> '$<'
<recognize output and generate makefile> > '$#'
Now when make runs it will see that foo.sym.mk is out of date (if it is) using normal algorithms and it will rebuild foo.sym.mk, which as a "side effect" causes the foo.s file to be compiled.
And of course, the "foo.sym.mk" file can include ANOTHER file, which can recognize the next step, if necessary.
I'm not saying this will be trivial but it seems do-able based on your description.
Make constructs the graph before running any rule, so there won't be a perfect answer. Here are some reasonably clean solutions.
1) use PHONY intermediates and wildcards in the commands. (You can't use Make wildcards because make expands them before running rules.)
final.results: middle
# build $# using $(shell ls foo.*.sym) and $(shell ls foo.*.hex)
.PHONY: middle
middle: foo.s
# build foo.XXXX.sym and foo.XXXX.hex from $<
2) Use recursive Make (which is not as bad as people say, and sometimes very useful.)
SYM = $(wildcard foo.*.sym)
HEX = $(wildcard foo.*.hex)
# Note that this is is the one you should "Make".
# I've put it first so it'll be the default.
.PHONY: first-step
first-step: foo.s
# build foo.XXXX.sym and foo.XXXX.hex from $<
#$(MAKE) -s final.results
final.results:
# build $# using $(SYM) and $(HEX)
3) Similar to 2, but have a rule for the makefile which will cause Make to run a second time.
SYM = $(wildcard foo.*.sym)
HEX = $(wildcard foo.*.hex)
final.results:
# build $# using $(SYM) and $(HEX)
Makefile: foo.s
# build foo.XXXX.sym and foo.XXXX.hex from $<
#touch $#
I am trying to use GNU AutoTools for my C++ project. I have written configure.ac, makefile.am etc. I have some files that are used by the program during execution e.g. template files, XML schema etc. So, I install/copy these files along the executable, for which I use something like:
abcdir = $(bindir)/../data/abc/
abc_DATA = ../data/knowledge/abc.cc
Now it copies the file correctly and My program installation structure looks somethings as follows:
<installation_dir>/bin/<executableFile>
<installation_dir>/data/abc/abc.cc
Now the problem is that in the source code I actually use these files (abc.cc etc.) and for that I need path of where these files resides to open them. One solution is to define (using AC_DEFINE) some variable e.g. _ABC_PATH_ that points to the path of installation but how to do that exactly?. OR is there any better way to do that. For example, in source code, I do something like:
...
ifstream input(<path-to-abc-folder> + "abc.cc"); // how to find <path-to-abc-folder>?
..
The AC_DEFINE solution is fine in principle, but requires shell-like variable expansion to take place. That is, _ABC_PATH_ would expand to "${bindir}/../data/abs", not /data/abc.
One way is to define the path via a -D flag, which is expanded by make:
myprogram_CPPFLAGS += -D_ABC_PATH='\"${abcdir}\"'
which works fine in principle, but you have to make include config.status in the dependencies of myprogram.
If you have a number of such substitution variables, you should roll out a paths.h file that is
generated by automake with a rule like:
paths.h : $(srcdir)/paths.h.in config.status
sed -e 's:#ABC_PATH#:${abcdir}:' $< > $#
As a side-note, you do know about ${prefix} and ${datarootdir} and friends, don't you? If not, better read them up; ${bindir}/.. is not necessarily equal to ${prefix} if the user did set ${exec_prefix}.
Is there a best practice for supporting dependencies on C/C++ preprocessor flags like -DCOMPILE_WITHOUT_FOO? Here's my problem:
> setenv COMPILE_WITHOUT_FOO
> make <Make system reads environment, sets -DCOMPILE_WITHOUT_FOO>
<Compiles nothing, since no source file has changed>
What I would like to do is have all files that rely on #ifdef statements get recompiled:
> setenv COMPILE_WITHOUT_FOO
> make
g++ FileWithIfdefFoo.cpp
What I do not want to is have to recompile everything if the value of COMPILE_WITHOUT_FOO has not changed.
I have a primitive Python script working (see below) that basically writes a header file FooDefines.h and then diffs it to see if anything is different. If it is, it replaces FooDefines.h and then the conventional source file dependency takes over. The define is not passed on the command line with -D. The disadvantage is that I now have to include FooDefines.h in any source file that uses the #ifdef, and also I have a new, dynamically generated header file for every #ifdef. If there's a tool to do this, or a way to avoid using the preprocessor, I'm all ears.
import os, sys
def makeDefineFile(filename, text):
tmpDefineFile = "/tmp/%s%s"%(os.getenv("USER"),filename) #Use os.tempnam?
existingDefineFile = filename
output = open(tmpDefineFile,'w')
output.write(text)
output.close()
status = os.system("diff -q %s %s"%(tmpDefineFile, existingDefineFile))
def checkStatus(status):
failed = False
if os.WIFEXITED(status):
#Check return code
returnCode = os.WEXITSTATUS(status)
failed = returnCode != 0
else:
#Caught a signal, coredump, etc.
failed = True
return failed,status
#If we failed for any reason (file didn't exist, different, etc.)
if checkStatus(status)[0]:
#Copy our tmp into the new file
status = os.system("cp %s %s"%(tmpDefineFile, existingDefineFile))
failed,status = checkStatus(status)
print failed, status
if failed:
print "ERROR: Could not update define in makeDefine.py"
sys.exit(status)
This is certainly not the nicest approach, but it would work:
find . -name '*cpp' -o -name '*h' -exec grep -l COMPILE_WITHOUT_FOO {} \; | xargs touch
That will look through your source code for the macro COMPILE_WITHOUT_FOO, and "touch" each file, which will update the timestamp. Then when you run make, those files will recompile.
If you have ack installed, you can simplify this command:
ack -l --cpp COMPILE_WITHOUT_FOO | xargs touch
I don't believe that it is possible to determine automagically. Preprocessor directives don't get compiled into anything. Generally speaking, I expect to do a full recompile if I depend on a define. DEBUG being a familiar example.
I don't think there is a right way to do it. If you can't do it the right way, then the dumbest way possible is probably the your best option. A text search for COMPILE_WITH_FOO and create dependencies that way. I would classify this as a shenanigan and if you are writing shared code I would recommend seeking pretty significant buy in from your coworkers.
CMake has some facilities that can make this easier. You would create a custom target to do this. You may trade problems here though, maintaining a list of files that depend on your symbol. Your text search could generate that file if it changed though. I've used similar techniques checking whether I needed to rebuild static data repositories based on wget timestamps.
Cheetah is another tool which may be useful.
If it were me, I think I'd do full rebuilds.
Your problem seems tailor-made to treat it with autoconf and autoheader, writing the values of the variables into a config.h file. If that's not possible, consider reading the "-D" directives from a file and writing the flags into that file.
Under all circumstances, you have to avoid builds that depend on environment variables only. You have no way of telling when the environment changed. There is a definitive need to store the variables in a file, the cleanest way would be by autoconf, autoheader and a source and multiple build trees; the second-cleanest way by re-configure-ing for each switch of compile context; and the third-cleanest way a file containing all mutable compiler switches on which all objects dependant on these switches depend themselves.
When you choose to implement the third way, remember not to update this file unnecessarily, e.g. by constructing it in a temporary location and copying it conditionally on diff, and then make rules will be capable of conditionally rebuilding your files depending on flags.
One way to do this is to store each #define's previous value in a file, and use conditionals in your makefile to force update that file whenever the current value doesn't match the previous. Any files which depend on that macro would include the file as a dependency.
Here is an example. It will update file.o if either file.c changed or the variable COMPILE_WITHOUT_FOO is different from last time. It uses $(shell ) to compare the current value with the value stored in the file envvars/COMPILE_WITHOUT_FOO. If they are different, then it creates a command for that file which depends on force, which is always updated.
file.o: file.c envvars/COMPILE_WITHOUT_FOO
gcc -DCOMPILE_WITHOUT_FOO=$(COMPILE_WITHOUT_FOO) $< -o $#
ifneq ($(strip $(shell cat envvars/COMPILE_WITHOUT_FOO 2> /dev/null)), $(strip $(COMPILE_WITHOUT_FOO)))
force: ;
envvars/COMPILE_WITHOUT_FOO: force
echo "$(COMPILE_WITHOUT_FOO)" > envvars/COMPILE_WITHOUT_FOO
endif
If you want to support having macros undefined, you will need to use the ifdef or ifndef conditionals, and have some indication in the file that the value was undefined the last time it was run.
Jay pointed out that "make triggers on date time stamps on files".
Theoretically, you could have your main makefile, call it m1, include variables from a second makefile called m2. m2 would contain a list of all the preprocessor flags.
You could have a make rule for your program depend on m2 being up-to-date.
the rule for making m2 would be to import all the environment variables ( and thus the #include directives ).
the trick would be, the rule for making m2 would detect if there was a diff from the previous version. If so, it would enable a variable that would force a "make all" and/or make clean for the main target. otherwise, it would just update the timestamp on m2 and not trigger a full remake.
finally, the rule for the normal target (make all ) would source in the preprocessor directives from m2 and apply them as required.
this sounds easy/possible in theory, but in practice GNU Make is much harder to get this type of stuff to work. I'm sure it can be done though.
make triggers on date time stamps on files. A dependent file being newer than what depends on it triggers it to recompile. You'll have to put your definition for each option in a separate .h file and ensure that those dependencies are represented in the makefile. Then if you change an option the files dependent on it would be recompiled automatically.
If it takes into account include files that include files you won't have to change the structure of the source. You could include a "BuildSettings.h" file that included all the individual settings files.
The only tough problem would be if you made it smart enough to parse the include guards. I've seen problems with compilation because of include file name collisions and order of include directory searches.
Now that you mention it I should check and see if my IDE is smart enough to automatically create those dependencies for me. Sounds like an excellent thing to add to an IDE.