Please explain this Makefile - executable rule appears to be missing - c++

I have this makefile which I don't understand.
build_sources:=$(wildcard *.cpp)
depends:=$(build_sources:.cpp=.d)
build_targets:=$(build_sources:.cpp=)
.PHONY: all
all: $(build_targets)
.PHONY: clean
clean:
rm -f $(build_targets) *.{a,o,d}
#build the list of header file dependencies automatically
%.d: %.cpp
#echo building include dependencies for $(*F)
#$(CXX) -MM $(CPPFLAGS) $< | { sed 's#\($*\)\.o[ :]*#\1.o $# : #g' ; echo "%.h:;" ; } > $#
-include $(depends)
I understand that the executables created are build_target. So if I have Foo.cpp and Bar.cpp, the executables created would be Foo and Bar.
But how does it do that? I only see 1 rule, and it is %.d: %.cpp. So it's saying the Foo.d file depends on Foo.cpp. But what about the rule to actually compile Foo?
This makefile works, so it's not actually missing anything. But how come I don't see the rule? Is there some implicit rule?
EDIT - i did make debug, and saw the following
No need to remake target `foo.d'.
Considering target file `foo'.
File `foo' does not exist.
Looking for an implicit rule for `foo'.
Trying pattern rule with stem `foo'.
Trying implicit prerequisite `foo.o'.
Found an implicit rule for `foo'. ## WHAT EXACTLY IS THIS?
Considering target file `foo.o'.
File `foo.o' does not exist.
Looking for an implicit rule for `foo.o'.
Trying pattern rule with stem `foo'.
Trying implicit prerequisite `foo.c'.
Trying pattern rule with stem `foo'.
Trying implicit prerequisite `foo.cc'.
Trying pattern rule with stem `foo'.
Trying implicit prerequisite `foo.C'.
Trying pattern rule with stem `foo'.
Trying implicit prerequisite `foo.cpp'.
Found an implicit rule for `foo.o'.
Pruning file `foo.cpp'.
Pruning file `foo.cpp'.
Finished prerequisites of target file `foo.o'.
Must remake target `foo.o'.
g++ -I../../include -Wall -std=c++11 -O3 -Wsign-compare -Wno-strict-aliasing -s -c -o foo.o foo.cpp
That's great. This must be the rule that's getting called. But where is this rule from? How do I know what default rules exist?

What you probably miss to understand this Makefile is the concept of pattern rule and automatic variables.
This part finds .cpp files:
build_sources:=$(wildcard *.cpp)
And this part creates targets with the same name (using the variable build_sources defined above) except the extension is replaced by .d:
depends:=$(build_sources:.cpp=.d)
The same kind of construction defines the build targets (same file name with extension removed):
build_targets:=$(build_sources:.cpp=)
Then the default target is defined to require build_targets, i.e the executable corresponding to the .cpp`
all: $(build_targets)
This rule defines how to build a .d from a .cpp:
#build the list of header file dependencies automatically
%.d: %.cpp
#echo building include dependencies for $(*F)
#$(CXX) -MM $(CPPFLAGS) $< | { sed 's#\($*\)\.o[ :]*#\1.o $# : #g' ; echo "%.h:;" ; } > $#
gcc -MM produce rules to know on which headers a .c or .cpp file depend. Typically if test.cpp includes test1.h and test2.h the output will be:
test.o: test.cpp test1.h test2.h
The .d will contain dependencies of each cpp file and will create a ruleto build a .o file and an executable from each cpp. $# is the target of the rule (the .d file) and will contain, unless I'm mistaken, a rule looking like this (written by the sed expression):
filename.o filename.d : filename.cpp <list of headers>
%.h:;
The first rule give dependencies of the .o files. It has no recipe which means it simply adds dependencies to any existing rule. An implicit rule will be used to build them.
The second one is here in case you suppress a header. In this case, make will use this rule that says that there's simply nothing to do (it has an empty recipe).
Then all the .d files are included as part of the makefile:
-include $(depends)
Finally, the implicit rule to link a single .o file kicks in:
Linking a single object file
n is made automatically from n.o by running the linker (usually called ld) via the C compiler. The precise recipe used is ‘$(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS)’.
Edit: To build objects in a subdirectory obj you would have to modify each filename:
depends:=$(foreach file,$(build_sources:.cpp=.d),"obj/$(file)")
To build binaries in a separate subdirectory bin you would need to do the same for build_targets:
build_targets:=$(foreach file,$(build_sources:.cpp=), "bin/$(file)")
Then you need to write the rule to build it because the default rule wont work anymore (the .o is not in the same directory). What you want is to add a rule like this:
bin/foo: obj/foo.o
$(CC) $(LDFLAGS) obj/foo.o $(LOADLIBES) $(LDLIBS) -o bin/foo
which can be done with the right modification of the long shell command:
#$(CXX) -MM $(CPPFLAGS) $< | { sed 's#\($*\)\.o[ :]*#\1.o $# : #g' ; echo "%.h:;" ; obj=`echo $< | sed 's/.cpp/.o/'` ; bin=`echo $< | sed 's/.cpp//'` ; echo "$bin: $obj" ; echo -e "\t\$(CC) \$(LDFLAGS) $obj \$(LOADLIBES) \$(LDLIBS) -o $bin" ; } > $#

Your make package includes a large number of default rules, which this Makefile depends on. Try running your make with -d (debug info). I believe that will show you everything that is in play.

Many rules are built in to Make. If you are using GNU Make, you can get it to print its built-in rules like this:
make -f /dev/null -p
You can make it ignore the built-in rules with -r; if you do this with your Makefile, you'll find that it complains that it doesn't know how to make your targets.

The makefile is't complete in itself. It automatically creates the files .d which are snippets included into that makefile at the end. These files are created with the %.d-rule.
So have a look at those generated .d files to see rules for each individual object file. The rule generation script is written with sed and kind of hard to read, but in fact very simple. At first, the compiler is invoked on the .cpp file with the -MM flag, which will output something like
foo.o: foo.cpp foo.h bar.h
if foo.cpp includes its own header file foo.h as well as bar.h, for example (directly or indirectly).
The sed regex replacement command now simply adds the name of the file, in which the generated rules are about to be written (the .d file) right after the .o in the above rule, so it is also marked as depending on the sources, just like the object file. That's important when the includes in one of the sources is changed later. The sed command also adds a rule for all header files to do nothing.
foo.o foo.d: foo.cpp foo.h bar.h
%.h:;
The resulting object files .o are then linked using one of the implicit rules:
Linking a single object file
n is made automatically from n.o by
running the linker (usually called ld) via the C compiler. The precise
recipe used is $(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS).
[...]

Related

Makefile to get source files in multiple subdirectories

I am creating a custom Makefile to to build a C++ Linux application. I have my cpp source files in a folder called src on the same level as the Makefile. So far I have been able to build my object files with the following :
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp | $(OBJ_DIR)
$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $#
Now my project is starting to get a bit more complicated and I want subdirectories within src , such as src/common , and also, not every file is a cpp file now but also a c file.
I guess having a separate Makefile for each subdirectory is the best way but I am trying to keep this simple for now with just one Makefile.
I tried the following but doesn't work
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp $(SRC_DIR)/common/%.cpp $(SRC_DIR)/common/%.c | $(OBJ_DIR)
$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $#
I could have the Makefile do a shell find to find all cpp and c files but also trying to avoid this.
I'd appreciate any recommendations.
This:
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp $(SRC_DIR)/common/%.cpp $(SRC_DIR)/common/%.c | $(OBJ_DIR)
cannot work; what it says that if make wants to build some file $(OBJ_DIR)/XXX.o and there is no explicit rule, then if and only if ALL the prerequisites $(SRC_DIR)/XXX.cpp, $(SRC_DIR)/common/XXX.cpp, and $(SRC_DIR)/common/XXX.c exist or can be created by make, then the rule will match.
If the same target could be built from multiple different prerequisites you must create multiple rules, one per prerequisite.
Also, it doesn't make sense to compile both C files (.c) and C++ files (.cpp) using the same recipe. C++ compilers use the variables CXX and CXXFLAGS and C compilers use the variables CC and CFLAGS.
As for avoiding find, you can't avoid informing make what files you want to be compiled, in some manner. You either have to list them in your makefile explicitly, or else use some method to generate them dynamically. There's no way around this. You don't have to use find if you don't want to; for example if you know that all the files will be either one or two directories down you could use wildcard, like:
SRCS := $(wildcard $(SRC_DIR)/*.cpp $(SRC_DIR)/*.c $(SRC_DIR)/*/*.cpp $(SRC_DIR)/*/*.c)

Building using Makefile: picks file.c instead of file.cpp even though specified explicitly

Abstract:
I have a .c file and a .cpp file with the same name in the same parent folder. I no longer wish to compile the .c file but doesn't want to remove it from the folder too.
Even though I explicitly specify to compile the .cpp file, the makefile was making use of the .c file. How can I fix this issue?
On renaming file3.c to another name, file3.cpp gets compiled.
Details:
The files present in 'src' folder are:
file1.c
file2.c
file3.c
file3.cpp
file4.cpp
Important extracts from the makefile:
SOURCE_FILES := ./src/file1.c \
./src/file2.c \
./src/file3.cpp \
./src/file4.cpp
OBJS = $(patsubst %.cpp,%.o,$(filter %.cpp, $(SOURCE_FILES))) \
$(patsubst %.c,%.o,$(filter %.c, $(SOURCE_FILES)))
all : $(OBJS)
$(CXX) $(CFLAGS) $(INCLUDES) -o $(PROJECT).bin $(OBJS) $(LDFLAGS) $(LIBRARIES)
.cpp.o:
$(CXX) $(CFLAGS) $(INCLUDES) -c $< -o $#
#echo 'CPP FILE : ' $<
.c.o:
$(CXX) $(CFLAGS) $(INCLUDES) -c $< -o $#
#echo 'C FILE : ' $<
On doing 'make', the log of the echo command gives:
C FILE : ./src/file1.c
C FILE : ./src/file2.c
C FILE : ./src/file3.c
CPP FILE : ./src/file4.cpp
Questions:
How can the above issue be solved?
Is there an easier way to fetch values for my OBJS variable?
The method make uses to pick which of two possible matching rules gets used depends on which version of make you are using. (It changed between 3.81 and 3.82 and I believe got tweaked slightly for 4.0+ but am not certain about that.)
I believe make 3.81 used the last matching rule and make 3.82+ uses the matching rule with the shortest stem (the bit that matches the %) and then the first matching rule between rules that have similar stem lengths.
So it would seem like you are using make 3.81 currently.
In that case it should be possible to simply swap the order of your two rules to get the behavior you want (assuming you always want .cpp files to "win").
Alternatively you could try giving make an explicit prerequisite for the file3.o target instead of letting it guess. (i.e. file3.o: file3.cpp)
That said neither of your explicit suffix rules is necessary as make already contains default rules to build .o files from both .c and .cpp files. (I don't know the relative ordering between them so you likely will need the explicit prerequisite, assuming that works, for the built-in rules to work correctly.)
Also you might want to look at using Pattern Rules instead of Old-Fashioned Suffix Rules just as a point of current practice.
The only improvement to your populating the OBJS variable that I can see, offhand, is to filter on the results and so just filter once.
OBJS = $(filter %.o,$(patsubst %.cpp,%.o,$(SOURCE_FILES)) \
$(patsubst %.c,%.o,$(SOURCE_FILES)))
That said you could also use Substitution References instead of explicit calls to $(patsubst) to shorten that a bit.
OBJS = $(filter %.o,$(SOURCE_FILES:.cpp=.o) $(SOURCE_FILES:.c=.o))
You have rules for both .c ==> .o and .cpp ==> .o.
In your case the rule for file3.c is run first since the newly created file3.o file now exists. Since the target file3.o file exists, there is no reason to compile file3.cpp.
Looking at the make manual, if there are 2 rules for the same target, the last rule is used.

Example makefile for building simple c project recompiling when headers change

Does anyone have a complete makefile that can do the following:
Rebuilds the project if a HEADER file changes
The cpp files are listed in the makefile
The header files are NOT listed in the makefile
The header files are allowed to have different names than the cpp files
Some of the cpp files do not have header files
I have seen instructions for figuring out how to make the make tool figure out header dependencies, but none of these instructions provide anything remotely resembling a working example. A simple example would be as follows: main.cpp C1.cpp C1.h C2.cpp C2.h
CXX = g++
OBJECTS := main.o C1.o C2.o
all: $(OBJECTS)
%.o : %.cpp
$(CXX) $(CPPFLAGS) -Wall -MMD -c $< -o $#
-include *.d
EDIT: As TobySpeight points out, this won't work if you build an object file, rename or delete one of the prerequisite source or header files, then try to rebuild the object file; the .d file will still require the missing file, and the build will fail. I neglected to include lines to deal with that case:
%.h: ;
%.cpp: ;
(This is effective, but crude. The more precise approach is to put some sed commands in the %.o rule, so as to add specific null rules to the .d file, one for each prerequisite, but the sed commands are ugly, and the approach above is good enough for almost all cases.)
You can also use CMake for this. Everything you need to write is:
add_executable (exec main.cpp C1.cpp C2.cpp)

proper usage of makefile

So, I'm trying to understand makefiles. At this moment I have a project with some modules, which are more or less non-dependent on each others.
My directory tree is like this:
root/
- Source/ <-- referenced as $(INPUTPATH)
- Build/ <-- referenced as $(BUILDPATH)
- Release/ <-- referenced as $(OUTPUTPATH)
- Makefile
I want to have the source (.cpp/.h) on the Source/, the objects on the Build/, and the executable on the Release/*.
So, my idea was to have several different targets, and use them like this:
all: maps auxiliars methods
#echo "linking...";\
$(CC) $(CFLAGS) $(OBJS) $(CLIBS) -o ${OUTPUTPATH}/MainProgram
dependency1: $(INPUTPATH)/foo.cpp
#echo "compiling ArbPrecision...";\
cd ${BUILDPATH};\
$(CC) $(CFLAGS) $(CINCLUDE) -c ../$?
dependency2: dependency1 $(INPUTPATH)/bar.cpp
#echo "compiling saddleConstructor...";\
cd ${BUILDPATH};\
$(CC) $(CFLAGS) $(CINCLUDE) -c ../$(INPUTPATH)/bar.cpp
maps: dependency2 $(INPUTPATH)/*Map.cpp
#echo "compiling maps...";\
cd ${BUILDPATH};\
$(CC) $(CFLAGS) $(CINCLUDE) -c ../$(INPUTPATH)/*Map.cpp
... (auxiliars and methods)
The $CINCLUDE and CFLAGS are just compiler stuff like headers and external libs.
Everything works almost perfect: on every target, it asks for other target's dependencies, and for the *cpp on the source. If this does not exist, it calls the target and compiles the *cpp. The executable is created without errors.
However, if I call the "make" twice, the time it takes to compile is the same for the first and second time, even if I don't make any changes on the source. So, apparently, I'm not putting the dependencies right.
Can anyone point me the right direction to perform this compilation? I'm doing something wrong and I'm not getting what it is.
Thanks,
Jorge
Make expects to operate off of files. For each rule, it checks to see if the target file named by that rule (1) exists and (2) is up to date (i.e., modified after any of its dependencies). If the file is not up to date, then it executes the rule to bring it up to date.
Therefore, as your makefile is currently written, when you run make, Make does the following:
all is the first rule, so if you don't specify any arguments, make processes the all rule.
No file named all exists, so the all rule needs to be executed.
all depends on the dependency1 rule. No file named dependency1 exists, so the dependency1 rule needs to be executed.
dependency1 depends on the dependency2 rule. No file named dependency2 exists, so the dependency2 rule needs to be executed.
What you instead want is something similar to the following:
OBJS := foo.o bar.o Map.o
all: $(OUTPUTPATH)/MainProgram
$(OUTPUTPATH)/MainProgram: $(OBJS)
$(CC) $(CFLAGS) $(OBJS) $(CLIBS) -o $(OUTPUTPATH)/MainProgram
%.o: %.cpp
$(CC) $(CFLAGS) $(CINCLUDE) $< -c $#
.PHONY: all
Note the following:
Since Make operates on files, rules' targets and dependencies are listed by filename wherever possible.
Rather than listing the .cpp files individually, an pattern rule is used to build each .o file. Within the pattern rule, the $< and $# automatic variables are used to specify the first prerequisite (file) and the target (file), respectively.
Make echoes its commands by default, so there's no need to echo yourself.
For GNU Make, all is listed as a phony target, to keep Make from looking for a file named all.

GCC dependency generation for a different output directory

I'm using GCC to generate a dependency file, but my build rules put the output into a subdirectory. Is there a way to tell GCC to put my subdirectory prefix in the dependency file it generates for me?
gcc $(INCLUDES) -E -MM $(CFLAGS) $(SRC) >>$(DEP)
I'm assuming you're using GNU Make and GCC. First add a variable to hold your list of dependency files. Assuming you already have one that lists all our sources:
SRCS = \
main.c \
foo.c \
stuff/bar.c
DEPS = $(SRCS:.c=.d)
Then include the generated dependencies in the makefile:
include $(DEPS)
Then add this pattern rule:
# automatically generate dependency rules
%.d : %.c
$(CC) $(CCFLAGS) -MF"$#" -MG -MM -MP -MT"$#" -MT"$(<:.c=.o)" "$<"
# -MF write the generated dependency rule to a file
# -MG assume missing headers will be generated and don't stop with an error
# -MM generate dependency rule for prerequisite, skipping system headers
# -MP add phony target for each header to prevent errors when header is missing
# -MT add a target to the generated dependency
"$#" is the target (the thing on the left side of the : ), "$<" is the prerequisite (the thing on the right side of the : ). The expression "$(<:.c=.o)" replaces the .c extension with .o.
The trick here is to generate the rule with two targets by adding -MT twice; this makes both the .o file and the .d file depend on the source file and its headers; that way the dependency file gets automatically regenerated whenever any of the corresponding .c or .h files are changed.
The -MG and -MP options keep make from freaking out if a header file is missing.
The answer is in the GCC manual: use the -MT flag.
-MT target
Change the target of the rule emitted by dependency generation. By default CPP takes the name of the main input file, deletes any directory components and any file suffix such as .c, and appends the platform's usual object suffix. The result is the target.
An -MT option will set the target to be exactly the string you specify. If you want multiple targets, you can specify them as a single argument to -MT, or use multiple -MT options.
For example, -MT '$(objpfx)foo.o' might give
$(objpfx)foo.o: foo.c
You may like this briefer version of Don McCaughey's answer:
SRCS = \
main.c \
foo.c \
stuff/bar.c
DEPS = $(SRCS:.c=.d)
Add -include $(DEPS) note the - prefix, which silences errors if the .d files don't yet exist.
There's no need for a separate pattern rule to generate the dependency files. Simply add -MD or -MMD to your normal compilation line, and the .d files get generated at the same time your source files are compiled. For example:
%.o: %.c
gcc $(INCLUDE) -MMD -c $< -o $#
# -MD can be used to generate a dependency output file as a side-effect of the compilation process.
Detailing on DGentry's answer, this has worked well for me:
.depend: $(SOURCES)
$(CC) $(CFLAGS) -MM $(SOURCES) | sed 's|[a-zA-Z0-9_-]*\.o|$(OBJDIR)/&|' > ./.depend
This also works in the case where there is only one dependency file that contains the dependency rules for all source files.
Ok, just to make sure I've got the question right: I'm assuming you have test.c which includes test.h, and you want to generate subdir/test.d (while not generating subdir/test.o) where subdir/test.d contains
subdir/test.o: test.c test.h
rather than
test.o: test.c test.h
which is what you get right now. Is that right?
I was not able to come up with an easy way to do exactly what you're asking for. However, looking at Dependency Generation Improvements, if you want to create the .d file while you generate the .o file, you can use:
gcc $(INCLUDES) -MMD $(CFLAGS) $(SRC) -o $(SUBDIR)/$(OBJ)
(Given SRC=test.c, SUBDIR=subdir, and OBJ=test.o.) This will create both subdir/test.o and subdir/test.d, where subdir/test.d contains the desired output as above.
If there is an argument to GCC to do this, I don't know what it is. We end up piping the dependency output through sed to rewrite all occurrences of <blah>.o as ${OBJDIR}/<blah>.o.
[GNU] make gets angry if you don't place the output in the current directory. You should really run make from the build directory, and use the VPATH make variable to locate the source code. If you lie to a compiler, sooner or later it will take its revenge.
If you insist on generating your objects and dependencies in some other directory, you need to use the -o argument, as answered by Emile.