C++ Makefile, is it possible to factorize it even more? - c++

Question

I have a project for school and I want to write a Makefile, I have seen some examples of using Makefile with multiple source directories and multiple executables but still could not implement it properly to my Makefile.
PS: I'm using doctest for the unit testing (and I can't change it).
Here is the project structure (and I can't change it):
.
├── bin
├── build
├── extern
│ └── doctest.h
├── include
│ ├── file1.hpp
│ └── file2.hpp
├── src
│ ├── file1.cpp
│ └── file2.cpp
├── tests
│ ├── file1-test.cpp
│ └── file2-test.cpp
└── Makefile
I have the following directories:
bin: for all the executables.
build: for all the objects (.o).
extern: for the doctest header (this is where I would have stored any other library)
include: for all the headers (.hpp).
src: for all the classes (.cpp).
tests: for all the unit tests (also .cpp)
You can see file1.cpp as a class, file1.hpp as the class header and file1-test.cpp as the unit tests for the class.
Here is my Makefile:
BIN_DIR := bin/
BUILD_DIR := build/
EXTERN_DIR := extern/
INCLUDE_DIR := include/
SOURCE_DIR := src/
TESTS_DIR := tests/
DEP_DIR := .dep/
DEPENDS := $(patsubst %.o, $(BUILD_DIR)$(DEP_DIR)%.d, $(notdir $(wildcard $(BUILD_DIR)*.o)))
EXE := $(addprefix $(BIN_DIR), file1-test file2-test)
OBJS_1 := $(addprefix $(BUILD_DIR), file1.o)
OBJS_2 := $(addprefix $(BUILD_DIR), file1.o file2.o)
CXX := clang++
CXXFLAGS := -Wall -std=c++11 -g -O3 -I$(INCLUDE_DIR) -I$(EXTERN_DIR)
vpath %.cpp $(SOURCE_DIR) $(TESTS_DIR)
.PHONY: all clean
all: $(EXE)
$(BUILD_DIR) $(BIN_DIR) $(BUILD_DIR)$(DEP_DIR):
#mkdir -p $#
$(BUILD_DIR)%.o: %.cpp | $(BUILD_DIR) $(BUILD_DIR)$(DEP_DIR)
#$(CXX) $(CXXFLAGS) -MMD -MP -MF $(BUILD_DIR)$(DEP_DIR)$(notdir $(basename $#).d) -c $< -o $#
$(BIN_DIR)%: $(BUILD_DIR)%.o | $(BIN_DIR)
#$(CXX) -o $# $^
$(BIN_DIR)file1-test: $(OBJS_1)
$(BIN_DIR)file2-test: $(OBJS_2)
.PRECIOUS: $(BUILD_DIR)%.o
-include $(DEPENDS)
clean:
-rm -rf $(BIN_DIR) $(BUILD_DIR)
So my questions are:
Is my Makefile following good practices ?
Is it optimized ? If no, how can I make it even better ?
For every new executable I've to add a OBJS_X variable and a target $(BIN_DIR)fileX-test: $(OBJS_X), can i get rid of it ? If yes can someone write me some generic rule, so I don't have to specify a variable and a target every time I want a new executable.
If I want to compile only one executable I have to use make bin/fileX-test. Is it possible to run only make fileX-test instead of make bin/fileX-test (but still building it in the bin directory) ? I tried to implement a rule like this: fileX-test: $(BIN_DIR)fileX-test but it's not working as I want, at the very end of the compilation it starts executing builtin rules and I don't know why. Can someone explain ?
Final answer:
This is what I considere a good answer, if it can help someone later:
BIN_DIR := bin/
BUILD_DIR := build/
EXTERN_DIR := extern/
INCLUDE_DIR := include/
SOURCE_DIR := src/
TESTS_DIR := tests/
DEP_DIR := $(BUILD_DIR).dep/
CXX := g++
CXXFLAGS := -Wall -std=c++11 -g -O3 -I$(INCLUDE_DIR) -I$(EXTERN_DIR)
DEPFLAGS := -MMD -MP -MF $(DEP_DIR)
vpath %.cpp $(SOURCE_DIR) $(TESTS_DIR)
file1-test_OBJECTS := $(addprefix $(BUILD_DIR), file1.o)
file2-test_OBJECTS := $(addprefix $(BUILD_DIR), file1.o file2.o)
EXE := $(patsubst %_OBJECTS, %, $(filter %_OBJECTS, $(.VARIABLES)))
.PHONY: all keep help check clean $(EXE)
all: $(EXE:%=$(BIN_DIR)%)
$(foreach E, $(EXE), $(eval $(BIN_DIR)$E: $($E_OBJECTS)))
$(foreach E, $(EXE), $(eval $E: $(BIN_DIR)$E ;))
$(BUILD_DIR) $(BIN_DIR) $(DEP_DIR):
#mkdir -p $#
$(BUILD_DIR)%.o: %.cpp | $(BUILD_DIR) $(DEP_DIR) $(BIN_DIR)
#$(CXX) $(CXXFLAGS) $(DEPFLAGS)$(#F:.o=.d) -c $< -o $#
$(BIN_DIR)%: $(BUILD_DIR)%.o
#$(CXX) -o $# $^
-include $(wildcard $(DEP_DIR)*.d)
keep: $(EXE:%=$(BUILD_DIR)%.o)
clean:
-#rm -rf $(BIN_DIR)* $(BUILD_DIR)* $(DEP_DIR)*

Answer 1

Mostly your makefile is pretty good. There are some simplifications you can make, but they're just syntax and not really performance etc.:
DEP_DIR := .dep/
You never use this by itself so if you change its definition to:
DEP_DIR := $(BUILD_DIR).dep/
you can simplify the references to it.
DEPENDS := $(patsubst %.o, $(BUILD_DIR)$(DEP_DIR)%.d, $(notdir $(wildcard $(BUILD_DIR)*.o)))
-include $(DEPENDS)
this seems complex. Why not get rid of DEPENDS and just write:
include $(wildcard $(DEP_DIR)*.d)
This:
#$(CXX) $(CXXFLAGS) -MMD -MP -MF $(BUILD_DIR)$(DEP_DIR)$(notdir $(basename $#).d) -c $< -o $#
is also complex. You can write it (if you simply DEP_DIR) as:
#$(CXX) $(CXXFLAGS) -MMD -MP -MF $(DEP_DIR)$(#F:.o=.d) -c $< -o $#
For:
.PRECIOUS: $(BUILD_DIR)%.o
I would definitely NOT use this. .PRECIOUS should be rarely, if ever, used. If you're trying to avoid object files being considered intermediate it's best to just list them directly as prerequisites, such as:
keep : $(EXE:$(BIN_DIR)%=$(BUILD_DIR)%.o)
But unless you have special need to look at these object files it doesn't hurt to let make delete them.
Regarding your question about shortcuts: the reason you see the behavior you do is that your target definition:
fileX-test: $(BIN_DIR)fileX-test
has no recipe attached to it, so make will try to find a recipe using an implicit rule. It finds built-in recipe for % : %.c, and because you set vpath it can find a %.c file that matches, so it uses it. To avoid this you can just give an empty recipe; replace the above with:
fileX-test: $(BIN_DIR)fileX-test ;
(note added semicolon).
Your main question is how to simplify this:
EXE := $(addprefix $(BIN_DIR), file1-test file2-test)
OBJS_1 := $(addprefix $(BUILD_DIR), file1.o)
OBJS_2 := $(addprefix $(BUILD_DIR), file1.o file2.o)
all: $(EXE)
$(BIN_DIR)file1-test: $(OBJS_1)
$(BIN_DIR)file2-test: $(OBJS_2)
You can do this automatically but doing so requires knowing the deeper parts of GNU make. You might find this set of blog posts interesting: http://make.mad-scientist.net/category/metaprogramming/ (start with the bottom / oldest and work your way up).
Replace the above with:
# Write one of these for each program you need:
file1-test_OBJECTS = file1.o
file2-test_OBJECTS = file1.o file2.o
# Now everything below here is boilerplate
EXE = $(patsubst %_OBJECTS,%,$(filter %_OBJECTS,$(.VARIABLES)))
all: $(EXE:%=$(BIN_DIR)%)
$(foreach E,$(EXE),$(eval $(BIN_DIR)$E: $($E_OBJECTS)))
$(foreach E,$(EXE),$(eval $E: $(BIN_DIR)$E ;))
.PHONY: $(EXE)

Answer 2

I am turning my comment into an answer to allow others to disapprove this view: I think CMake is better here for you. Look at this SO for some differences between Make and CMake and arguments for CMake.
Advantages related to your questions:
It will allow you more easily to follow good practices
It scales much better
You do not have to write so muc boilerplate for new executable added to your code
Building a single executable is possible, see this SO as a hint.