My C++ project has source files organized in nested subdirectories of ./src. I have a pattern rule in my makefile which compiles all of the .cpp source files into objects:
$(OBJDIR)/%.o: %.cpp makefile
$(CXX) -c $< -o $#
Since I am using this pattern rather than writing a compilation rule for each source file, I need to tell make to look recursively through ./src for these prerequisites. Right now I have:
VPATH := $./src/:./src/folder1:./src/folder2:./src/folder3
This works, but it feels pretty inelegant and also causes bugs when I inevitably forget to add in a new folder.
Hoping someone has a better solution!
You can automate the building of the VPATH variable like yours by searching for subdirectories and replacing spaces with colons:
space :=
space +=
VPATH := $(subst $(space),:,$(shell find src -type d))
This assumes that you have no spaces in your directories or filenames.
With this approach, it is not clear to me what you would do if two source files in two different subdirectories have the same name -- but that seems to be more related to your overall setup than to your question about the VPATH specifically.
For the $(space) variable trick, see the nifty Escaping comma and space in GNU Make blog post.
gmake itself does not have any functions for recursive directory traversal, so you have to resort to $(shell ...):
VPATH := $(shell find src -type d -print | tr '\012' ':' | sed 's/:$$//')
Tweak the shell script to get the right semantics. You want to use the := operator, in order to evaluate this one time.
Related
I want to use an awk script to figure out the which modules have to be compiled before I can compile a FORTRAN source file. My project is structured in a way that I can obtain the filenames that provide the modules by running
awk '$1=/use/{print gensub(",","","g", $2) ".o"}' file.f90
on the file I want to compile.
However, my make command
%.o: $(shell awk '$$1=/use/{print gensub(",","","g", $$2) ".o"}' /path/to/%.f90)
fails with
awk: fatal: cannot open file `/path/to/%.f90' for reading: No such file or directory
So %.f90 does not get expanded. Why is that the case and how can I solve that issue?
Variables and functions in targets and prerequisites are expanded when the makefile is parsed, not when make is running the makefile. But, pattern rules are only expanded when make is running the makefile, trying to build a target that matches the pattern. So at the time these variables and functions are expanded, you only have the literal pattern string not its expansion into a real filename.
See How make reads a makefile in the docs.
There are a number of ways to do this. One option is using secondary expansion. However note you'll have to double-escape the $ you are escaping!!
.SECONDEXPANSION:
%.o: $$(shell awk '$$$$1=/use/{print gensub(",","","g", $$$$2) ".o"}' /path/to/$$*.f90)
ETA
You could alternatively not use .SECONDEXPANSION at all and instead use eval like this:
%.o:
...
SRCS := $(wildcard *.f90)
OBJS := $(SRCS:%.f90=%.o)
$(foreach O,$(OBJS),\
$(eval $O: $(shell awk '$$1=/use/{print gensub(",","","g", $$2) ".o"}' $(O:%.o=%.f90))))
Since you didn't give an actual example I just made up SRCS and OBJS variables. Maybe you have similar variables already.
I'm using GNU makefiles to build a C project. I want to keep all build artifacts on an isolated build tree in order to minimize clutter. The project looks like this:
$prefix/
include/$tree/program.h
source/$tree/program.c
build/
objects/$tree/program.o
dependencies/$tree/program.d
Where $prefix represents the directory of the project and $tree represents an arbitrary folder structure.
I wanted to match source files in the source/ directory to their object and dependency file counterparts in the build/ tree. So, I wrote the following rules:
# Remove the built-in rules
%.o : %.c
$(objects_directory)/%.o : $(source_directory)/%.c $(dependencies_directory)/%.d
$(compiler_command_line) $(compiler_option_output) $# $<
$(build_directory)/$(target) : $(objects)
$(compiler_command_line) $(compiler_option_output) $# $^
Make correctly figures out the compilation target and the object files needed to build it. However, make stops at this point with the error:
No rule to make target 'build/objects/project/program.o', needed by 'build/program.dll'.
So why is this happening, and how do I fix it?
I investigated the problem by running make --print-data-base, the output of which included:
# Not a target:
build/objects/project/program.o:
# Implicit rule search has been done.
# File does not exist.
# File has not been updated.
Which suggests that the prerequisite is not matching the implicit rule as intended. However, I verified that it does match when I tried to work my way backwards by writing:
object := build/objects/project/program.o
$(object:$(objects_directory)/%.o=$(source_directory)/%.c)
$(object:$(objects_directory)/%.o=%)
These lines result in source/project/program.c and project/program, which means the stem is being correctly computed.
I have studied the GNU make documentation and I don't remember reading anything that suggests that this kind of pattern matching can't happen in implicit rule definitions.
Here are the variable definitions:
include_directory := include
source_directory := source
build_directory := build
objects_directory := $(build_directory)/objects
dependencies_directory := $(build_directory)/dependencies
sources := $(wildcard $(source_directory)/**/*.c)
objects := $(sources:$(source_directory)/%.c=$(objects_directory)/%.o)
# Take the name of the project's directory
target := $(notdir $(CURDIR)).dll
compiler_dependency_file = $(patsubst $(source_directory)/%.c,$(dependencies_directory)/%.d,$<)
compiler_options = -I $(include_directory) -MMD -MF $(compiler_dependency_file)
CC = gcc
compiler_command_line = $(CC) $(compiler_options) $(CFLAGS)
compiler_option_output = -o
It turns out it wasn't the pattern matching. The root of the problem was in the dependency prerequisite of the implicit rule.
The dependency file isn't supposed to be a prerequisite in the first place; it should be one of the targets that gets generated along with the object file.
As I read once more ยง 4.14 Generating Prerequisites Automatically of the manual, the answer jumped out at me:
The purpose of the sed command is to translate (for example):
main.o : main.c defs.h
into:
main.o main.d : main.c defs.h
While my build system makes no use of sed, the fact that main.d was on the left-hand side of the example rule felt strange. In my code, it was on the right-hand side.
When I put my rule in the left-hand side, it worked and the problem was solved. The erroneous recipe was essentially treating one of its byproducts as a prerequisite.
In my project all .cpp files are stored in
Classes/
Classes/Something/
Classes/Something/Else
...
I want to compile all .cpp file separetly to Bin/ directory, replacing / with _, so that:
Classes/First.cpp -> Bin/Classes_First.o
Classes/Foo/Bar.cpp -> Bin/Classes_Foo_Bar.o
Now I wanted to create rules for compiling:
Bin/%.o: $(subst _,/,%.cpp)
$(COMPILER)g++ $(COMPILE_FLAGS) -c -o $# $^
I tried:
make Bin/Classes_Test.o
But compilation failed.
So I created debugging pattern:
%.cpp:
#echo CPP: $#
Now it printed:
CPP: Classes_Test.cpp
Why?!
So I changed my pattern to:
Bin/%.o: $(subst _,/,Test1_Test2.cpp)
and I saw:
CPP: Test1/Test2.cpp
I'm a little bit confused why subst does not work if I use wildcard as source...
This is an evaluation order issue.
When make parse the makefile it evaluates the $(subst) call but the argument to $(subst) at that point is the literal string %.cpp which has nothing to substitute in it and so does not do anything.
At target evaluation/execution time the % in the target pattern and prereq pattern are filled out but the $(subst) has long-since gone away.
To do this you will need to manually (in one way or another) map the output files to the input files. You can do that and keep the %.o pattern rule target for the actual recipe to run though (so you just need to generate a bunch of Bin/Test1_Test2.o: Test1/Test2.cpp lines).
Alternatively, I believe you can might be able to use secondary expansion to do this:
.SECONDEXPANSION:
Bin/%.o: $$(subst _,/,%.cpp)
CXXSRC = $(shell find source -iname "*.cpp")
CXXSRCFN = $(notdir $(CXXSRC))
CXXOBJ = $(CXXSRCFN:%.s=output/obj/%.cpp.o)
OUTPUT = output/kernel.elf
.PHONY: builduserspace clean all
all: $(OUTPUT)
#$(QEMU) -vga std -serial file:"output/serialout.log" -m 20 -hda output/disk.img -rtc base=localtime
$(OUTPUT): $(CXXOBJ)
# Linking object files...
#$(LD) $(LDFLAGS) -o output/temp.elf output/obj/Start.s.o $(shell find output/obj -name "*.o" ! -name "Start.s.o") -Lgcc
# Performing objcopy...
#$(OBJCOPY) -O elf32-i386 output/temp.elf output/kernel.elf
%.cpp.o: %.cpp
#echo $(notdir $<)
#$(CXX) $(CXXFLAGS) -o $(notdir $#) $<
That's the makefile. Here's the situation, imagine this directory structure:
output
|
|-- obj
source
|
|-- file1.cpp
|-- file2.cpp
|-- subdirectory
|
|-- file3.cpp
Say I run make in the root folder (where output and source are). The target output is output/kernel.elf.
Essentially, I want all the files in the folder 'source' to be compiled into object files and placed into the folder output/obj.
I managed to get the variables about right; CXXSRC is simply the list of all source files to be compiled; CXXOBJ is the list of outputs.
However, make: * No rule to make target output/obj/file.cpp.o', needed byoutput/kernel.elf'. Stop.
After some trial and error, I managed to narrow down the issue: If I modify the target to this:
output/obj/%.cpp.o: source/subdirectory/%.cpp
It works fine (ie. it errors on the other files in my tree, because not all files are in subdirectory)
Obviously this would defeat the purpose of the % wildcard operator, so how do I fix this problem?
Thanks.
First SO post, take it easy on me (:
The short answer is, you can't. In a pattern rule the pattern (%) must be identical (string-wise) between the target and the prerequisite. It can't mean "something sort of, but not exactly, the same".
I think it's a bit unpleasant to compile source files from multiple different directories and put the output into a single directory. Every single time I've seen that done it's become a big problem (because people sometimes use the same source file name and then you have a mess).
If you really want to do it there's no choice but to declare multiple rules with the different source directories. There are ways to do this in the makefile without writing them all by hand, but they're somewhat more advanced.
I use GNU make for building reports (LaTeX for source, python for figures, etc.). For targets, I use extensively pattern matching, for example:
all : sample1_test1.png sample2_test1.png sample2_test.png
sample%_test1.png : sample%.dat
python gen_figure.py $< $# --test 1
sample%_test2.png : sample%.dat
python gen_figure.py $< $# --test 2
Now, to simplify the rules I would like to use multiple pattern groups (like regex groups) and use the matches separately in the build rule, for example:
all : sample1_test1.png sample2_test1.png sample2_test.png
sample(?P<Sample>[0-9]+)_test(?P<Test>[0-9]+).png : sample$(<Sample>).dat
python gen_figure.py $< $# --test $(<Test>)
(the syntax is arbitrary, but the point is that I can define two different match groups called Sample and Test and use them as parameters to my script).
How would I achieve this in make or another build system (waf, scons etc.)?
To do it in GNU make, you can use one of two different "metaprogramming" models supported by GNU make:
Auto-generated include files. In your main makefile, add -include generated.mk then write a makefile rule with the target generated.mk (probably listing Makefile as a prerequisite), where the recipe generates the appropriate targets based on the list of targets. You have the full power of the shell to construct your target lists however you want. Every time you modify the makefile, the included file will be rebuilt then GNU make will automatically re-exec itself so you don't have to do anything extra.
Use GNU make's $(eval ...) function, probably combined with $(call ...) and $(foreach ...), to automatically evaluate rules. To do this you define a "template" for the rule using define ... enddef, with variables installed where you want to provide arguments, then use $(call ...) to instantiate them, use $(eval ...) on the result of the call, and do it in a loop for each target. Something like: $(foreach T,$(TARGETS),$(eval $(call DEFINERULE,$(T))))
Here's an example of method 1. Suppose you have this predefined content in your makefile:
TESTS := sample1_test1.png sample2_test1.png sample2_test.png
Then you can use this makefile to get something like the above:
all: $(TESTS)
-include generated.mk
generated.mk : Makefile
#rm -f '$#'
#for t in $(TESTS); do \
eval `echo "$$t" | sed 's/^sample\([0-9]*\)_test\([0-9]*\).*/sample=\1 test=\2/'`; \
echo "$$t : sample$$sample.dat ; python gen_figure.py \$$< \$$# --test $$test" >> '$#'; \
done
Note I just wrote this off the top of my head but I think it will work.