C++ Makefile with source in multiple directories and multiple executables - c++

I have a small project for school and I want to write a decent Makefile, I have seen some examples of using Makefile with multiple source directories and multiple executables but still could not implement it properly to my Makefile.
Here is the directory structure:
.
├── bin (this is where i want the executables)
├── build (this is where i want the .o)
├── extern
│ └── doctest.h
├── include
│ ├── file1.hpp
│ └── file2.hpp
├── src
│ ├── file1.cpp
│ └── file2.cpp
├── tests
│ ├── file1-test.cpp
│ └── file2-test.cpp
└── Makefile
PS: I'm using doctest for the unit testing (and i can't change).
Here is my Makefile for the moment:
BIN_DIR = ./bin/
BUILD_DIR = ./build/
EXTERN_DIR = ./extern/
INCLUDE_DIR = ./include/
SOURCE_DIR = ./src/
TESTS_DIR = ./tests/
CXX = clang++-7
CXXFLAGS = -Wall -std=c++11 -g -O3 -I$(INCLUDE_DIR)
EXEC_FILES = file1 file2
BIN = $(addprefix $(BIN_DIR), $(EXEC_FILES))
all: $(BIN)
$(BIN_DIR) $(BUILD_DIR):
mkdir -p $#
$(BUILD_DIR)%.o: $(SOURCE_DIR)%.cpp | $(BUILD_DIR)
$(CXX) $(CXXFLAGS) -c -o $# $^
$(BUILD_DIR)%.o: $(TESTS_DIR)%.cpp | $(BUILD_DIR)
$(CXX) $(CXXFLAGS) -I$(EXTERN_DIR) -c -o $# $^
$(BIN_DIR)file1-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file1-test.o | $(BIN_DIR)
$(CXX) -o $# $^
$(BIN_DIR)file2-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file2.o $(BUILD_DIR)file2-test.o | $(BIN_DIR)
$(CXX) -o $# $^
clean:
rm -f $(BIN_DIR)* $(BUILD_DIR)*
It's working well but I feel like it's doing useless stuff that i could avoid with more knownledge in the Makefile art, sadly I don't have much time.
My goal:
I want to run make and compile all the units tests (all the .cpp in the tests/ directory) and the main program (see PS below).
I want all the executables in the bin/ directory and all the binary files in the build/ directory.
PS: I don't have my main executable yet but it should be stored in the src/ directory at the very end (and be compiled as well while running make).
EDIT 1:
Basically, my question is: How can I write generic rules to prevent copying this (see below) for every new executable in my project ?
$(BIN_DIR)file2-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file2.o $(BUILD_DIR)file2 test.o | $(BIN_DIR)
$(CXX) -o $# $^
EDIT 2:
After some changes, I ended up with this:
BIN_DIR = ./bin/
BUILD_DIR = ./build/
EXTERN_DIR = ./extern/
INCLUDE_DIR = ./include/
SOURCE_DIR = ./src/
TESTS_DIR = ./tests/
vpath %.cpp $(SOURCE_DIR) $(TESTS_DIR)
CXX = clang++-7
CXXFLAGS = -Wall -std=c++11 -g -O3 -I$(INCLUDE_DIR) -I$(EXTERN_DIR)
EXEC_FILES = file1-test file2-test
BIN = $(addprefix $(BIN_DIR), $(EXEC_FILES))
all: $(BIN) | $(BIN_DIR)
$(BUILD_DIR)%.o: %.cpp | $(BUILD_DIR)
#$(CXX) $(CXXFLAGS) -c -o $# $^
# The problem is here
$(BIN_DIR)%: $(BUILD_DIR)%.o
#$(CXX) -o $# $^
$(BIN_DIR) $(BUILD_DIR):
#mkdir -p $#
clean:
-#rm -f $(BIN_DIR)* $(BUILD_DIR)*
I understand this rule (see below) like this: Compile every target in EXEC_FILES to the bin/ directory, from his equivalent .o (obj) in the build/ directory.
$(BIN_DIR)%: $(BUILD_DIR)%.o
#$(CXX) -o $# $^
But in fact I need a rule that will: Compile every target in EXEC_FILES to the bin/ directory, from all their related .o (obj) in the build/ directory.

Related

C++ Makefile auto-depencies with multiple executables

I have a project for school and I want to write a Makefile, I have seen some examples of using Makefile with multiple source directories and multiple executables but still could not implement it properly to my Makefile.
PS: I'm using doctest for the unit testing (and I can't change it).
Here is the project structure (and I can't change it):
.
├── bin
├── build
├── extern
│ └── doctest.h
├── include
│ ├── file1.hpp
│ └── file2.hpp
├── src
│ ├── file1.cpp
│ └── file2.cpp
├── tests
│ ├── file1-test.cpp
│ └── file2-test.cpp
└── Makefile
I have the following directories:
bin: for all the executables.
build: for all the objects (.o).
extern: for the doctest header (this is where I would have stored any other library)
include: for all the headers (.hpp).
src: for all the classes (.cpp).
tests: for all the unit tests (also .cpp)
You can see file1.cpp as a class, file1.hpp as the class header and file1-test.cpp as the unit tests for the class.
In the exemple above I have 2 tests files but at the very end of the project I'll have a lot more, and for each test file I'll have an executable.
My goals:
I want to run make and compile all the units tests (all the .cpp in the tests/ directory).
And I want all the executables to be stored in the bin/ directory and all the binary files in the build/ directory.
Here is my Makefile:
BIN_DIR = ./bin/
BUILD_DIR = ./build/
EXTERN_DIR = ./extern/
INCLUDE_DIR = ./include/
SOURCE_DIR = ./src/
TESTS_DIR = ./tests/
vpath %.cpp $(SOURCE_DIR) $(TESTS_DIR)
CXX = clang++
CXXFLAGS = -Wall -std=c++11 -g -O3 -I$(INCLUDE_DIR) -I$(EXTERN_DIR)
EXEC_FILES = file1-test file2-test
BIN = $(addprefix $(BIN_DIR), $(EXEC_FILES))
all: $(BIN) | $(BIN_DIR)
$(BUILD_DIR)%.o: %.cpp | $(BUILD_DIR)
$(CXX) $(CXXFLAGS) -c -o $# $^
$(BIN_DIR) $(BUILD_DIR):
mkdir -p $#
$(BIN_DIR)file1-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file1-test.o
$(CXX) -o $# $^
$(BIN_DIR)file2-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file2.o $(BUILD_DIR)file2-test.o
$(CXX) -o $# $^
clean:
-rm -f $(BIN_DIR)* $(BUILD_DIR)*
It's working well but I feel like it's doing redondant stuff that i could avoid with more knownledge in the Makefile art, especially here:
$(BIN_DIR)file1-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file1-test.o
$(CXX) -o $# $^
$(BIN_DIR)file2-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file2.o $(BUILD_DIR)file2-test.o
$(CXX) -o $# $^
For the moment this Makefile is correct because I only have 2 executables, but I'll end up with 15+ and I dont want to have 15 times this for each executable:
$(BIN_DIR)xxx-test: $(BUILD_DIR)xxx.o etc.
$(CXX) -o $# $^
What I exactly need ...:
Basically, I need to write a generic rule that will fetch all the appropriated dependencies for a given target.
After reading multiple posts I think it's all about auto-dependencies.
I'm pretty sure the final result would look like this, but sadly I can't make it works in my case:
$(BIN_DIR)%: ???
#$(CXX) -o $# $^
I already looked at this (and many other posts about the subject): http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/, but I still can't figure it out.
So how can I write an expression that will do the job, can someone give me a working exemple or something similar ?
EDIT 1:
Based on this post: Makefile (Auto-Dependency Generation).
I added these lines to my Makefile:
SRC = $(wildcard $(SOURCE_DIR)*.cpp)
SRC += $(wildcard $(TESTS_DIR)*.cpp)
The idea is to fetch all the .cpp from the source directories (src and tests). Then I added -MDD option to my CXXFLAGS variable to create a .d file for each target (atleast it's what I thought it's doing):
CXXFLAGS = -Wall -std=c++11 -g -O3 -I$(INCLUDE_DIR) -I$(EXTERN_DIR) -MMD
And finally, I added this:
$(BIN_DIR)%: $(SRC)
$(CXX) -o $# $^
-include $(SRC:.cpp=.d)
What I expect it to do:
Create a .d file with all the dependencies for each target.
Fetch the dependencies in the .d file and transform them to .o to get all the objects needed for the given target.
But it seems that it's not doing what I'm expecting.
EDIT 3:
After some changes I end up with this Makefile:
BIN_DIR := bin/
BUILD_DIR := build/
EXTERN_DIR := extern/
INCLUDE_DIR := include/
SOURCE_DIR := src/
TESTS_DIR := tests/
DEP_DIR := .dep/
DEPENDS := $(patsubst %.o, $(BUILD_DIR)$(DEP_DIR)%.d, $(notdir $(wildcard $(BUILD_DIR)*.o)))
EXE := $(addprefix $(BIN_DIR), Coord-test Fourmi-test)
OBJS_1 := $(addprefix $(BUILD_DIR), Coord.o)
OBJS_2 := $(addprefix $(BUILD_DIR), Coord.o Fourmi.o)
CXX := clang++
CXXFLAGS := -Wall -std=c++11 -g -O3 -I$(INCLUDE_DIR) -I$(EXTERN_DIR)
vpath %.cpp $(SOURCE_DIR) $(TESTS_DIR)
all: $(EXE)
$(BUILD_DIR):
mkdir -p $# $#/$(DEP_DIR)
$(BIN_DIR):
mkdir -p $#
$(BUILD_DIR)%.o: %.cpp | $(BUILD_DIR)
$(CXX) $(CXXFLAGS) -MMD -MP -MF $(BUILD_DIR)$(DEP_DIR)$(notdir $(basename $#).d) -c $< -o $#
$(BIN_DIR)%: $(BUILD_DIR)%.o | $(BIN_DIR)
$(CXX) -o $# $^
$(BIN_DIR)Coord-test: $(OBJS_1)
$(BIN_DIR)Fourmi-test: $(OBJS_2)
.PRECIOUS: $(BUILD_DIR)%.o
-include $(DEPENDS)
clean:
-rm -f $(BIN_DIR)* $(BUILD_DIR)* $(BUILD_DIR)$(DEP_DIR)*
It's working but I'll have to add OBS_X for each new executable.
I also wanted factorize this, but I don't know if it's possible ? If someone could tell me.
$(BIN_DIR)%: $(BUILD_DIR)%.o | $(BIN_DIR)
$(CXX) -o $# $^
$(BIN_DIR)Coord-test: $(OBJS_1)
$(BIN_DIR)Fourmi-test: $(OBJS_2)
Since you know that you will always have a foo-test.o to build a foo-test program, you can write your pattern rule like this:
$(BIN_DIR)%: $(BUILD_DIR)%.o
$(CXX) -o $# $^
However, there's no way make can infer what OTHER objects might be needed to build these executables. You'll just have to tell it. So for the above examples you can add this:
$(BIN_DIR)file1-test: $(BUILD_DIR)file1.o
$(BIN_DIR)file2-test: $(BUILD_DIR)file1.o $(BUILD_DIR)file2.o
You don't need to put the recipe here, this is just adding more prerequisites to these targets. You also don't have to put in the $(BUILD_DIR)file1-test.o etc. because this is inferred from the pattern rule.
But, if you do have other object files you need to use you'll have to list them explicitly, there's no way around it.

makefile: pattern rules for subdirectory

Here is my project:
project
|--- main.cpp
|--- makefile
|--- test
|--- Test.cpp
|--- Test.h
Here is the makefile:
g++1x:=g++ -std=c++14 -stdlib=libc++ -MMD -MP
cflags:= -Wall -lncurses
PATHS:=./ ./test/
TARGET:=matrix.out
SRC:=$(foreach PATH,$(PATHS),$(wildcard $(PATH)/*.cpp))
OBJDIR:=.obj
OBJ:=$(addprefix $(OBJDIR)/,$(notdir $(SRC:.cpp=.o)))
.PHONY: install
install: $(OBJDIR) $(TARGET)
$(OBJDIR):
mkdir -p $(OBJDIR)
$(TARGET): $(OBJ)
$(g++1x) $(cflags) -o $# $^ -g
$(OBJDIR)/%.o: %.cpp
$(g++1x) -c -o $# $< -g
$(OBJDIR)/%.o: ./test/%.cpp
$(g++1x) -c -o $# $< -g
-include $(addprefix $(OBJDIR)/,$(notdir $(SRC:.cpp=.d)))
.PHONY: clean
clean:
rm -f $(TARGET)
rm -rf $(OBJDIR)
It works well but I have two questions:
1) Is it possible to avoid foreach for PATHS so that I can use the same makefile for all of cpp projects?
2) As you see, to generate main.o and Test.o I write two blocks:
$(OBJDIR)/%.o: ./test/%.cpp and $(OBJDIR)/%.o: %.cpp.
Is it possible to write only once?
I've tried as below but it doesn't work:
$(OBJDIR)/%.o: $(foreach PATH,$(PATHS),$(wildcard $(PATH)/%.cpp))
$(g++1x) -c -o $# $< -g
I've even tried like this but it doesn't work still:
$(OBJDIR)/%.o: %.cpp ./test/%.cpp
$(g++1x) -c -o $# $< -g
You should keep the source tree into your object tree. This way it will be easier to create global rules and keep dependencies.
# Use the shell find command to get the source tree
SOURCES := $(shell find * -type f -name "*.c")
OBJDIR := .objects
# Keep the source tree into the objects tree
OBJECTS := $(addprefix $(OBJDIR)/,$(SOURCES:.c=.o))
all: mytarget
mytarget: $(OBJECTS)
$(CC) $^ -o $#
# As we keep the source tree we have to create the
# needed directories for every object
$(OBJECTS): $(OBJDIR)/%.o: %.c
mkdir -p $(#D)
$(CC) -MMD -MP -c $< -o $#
-include $(OBJECTS:.o=.d)
$ make
mkdir -p .objects
cc -MMD -MP -c main.c -o .objects/main.o
mkdir -p .objects/test
cc -MMD -MP -c test/test.c -o .objects/test/test.o
cc .objects/main.o .objects/test/test.o -o mytarget
$ tree -a
.
├── main.c
├── Makefile
├── mytarget
├── .objects
│   ├── main.d
│   ├── main.o
│   └── test
│   ├── test.d
│   └── test.o
└── test
├── test.c
└── test.h
3 directories, 9 files
EDIT: You can reduce the number of mkdir to the minimum by adding the object directory as an order only prerequisites:
# Do not create the directory
$(OBJECTS): $(OBJDIR)/%.o: %.c
$(CC) -MMD -MP -c $< -o $#
# Every target finishing with "/" is a directory
$(OBJDIR)%/:
mkdir -p $#
# Add the "directory/" as an order only prerequisite
$(foreach OBJECT,$(OBJECTS),$(eval $(OBJECT): | $(dir $(OBJECT))))
Because I can't edit jmlemetayer's answer, here's how to extend the commands for projects with both C and CPP source:
Firstly to generate the object list, just apply the patsubst twice, once for each extension:
SRC_LST := $(patsubst %.c,%.o,$(patsubst %.cpp,%.o,$(SOURCES)))
OBJECTS := $(addprefix $(OBJDIR)/,$(SRC_LST))
Then to handle both .cpp and .c files in the implicit rule for compiling:
.SECONDEXPANSION:
$(OBJECTS): $(OBJDIR)/%.o: $$(wildcard %.c*)
mkdir -p $(#D)
ifeq "$(suffix $<)" ".cpp"
$(CPP) -MMD -MP -c $< -o $#
else
$(CC) -MMD -MP -c $< -o $#
endif
If there is a better way I'm happy to update this.

makefile compiles all files instead of the last edited ones

Note that there are similar questions on SO, however I think my situation is different, moreover my Makefile is extremely simple and straight forward. I am new to Makefile.
Suppose I need to compile a project, that looks like this --
.
├── [4.0K] bin
├── [ 517] Makefile
├── [4.0K] obj
└── [4.0K] src
├── [ 117] func1.cpp
├── [ 76] func2.cpp
├── [ 137] global.h
└── [ 97] main1.cpp
and my Makefile looks like this --
CC := g++
CFLAGS := -g -Wall -std=c++0x
LDFLAGS := -lm
NAMES := func1.cpp func2.cpp main1.cpp
SRC := $(addprefix src/,$(NAMES))
OBJ := $(addprefix obj/,$(NAMES:.cpp=.o))
DEPS := $(OBJ:.o=.d)
.PHONY: clean all debug
all: prog
debug:
$(info $$SRC: ${SRC})
$(info $$OBJ: ${OBJ})
$(info $$DEPS: ${DEPS})
prog: bin/prog
bin/prog: $(OBJ)
$(CC) $^ -o $#
$(OBJ): $(SRC)
$(CC) $(CFLAGS) -I/src/global.h -c $(addprefix src/,$(notdir $(#:.o=.cpp))) -o $#
-include $(DEPS)
clean:
rm -rf bin/*
rm -rf obj/*
Suppose I opened a file func1.cpp and made some changes. When I invoke make it compiles all files, but it was supposed to compile only one (func1.cpp).
How do I fix this ?
Note: I need prog, bin/prog for a different reason, also I can't do recipes like obj/%.o: src/%.c because I might have different target from the subset of the same objects.
When you write a rule like:
$(OBJ): $(SRC)
cmd
which in your case is
obj/func1.o obj/func2.o obj/main.o : src/func1.cpp src/func2.cpp src/main1.cpp
cmd
The prerequisites don't get zipped across. That generates one rule for each target, with all of the prerequisites. That is:
obj/func1.o : src/func1.cpp src/func2.cpp src/main1.cpp
cmd
obj/func2.o : src/func1.cpp src/func2.cpp src/main1.cpp
cmd
obj/main.o : src/func1.cpp src/func2.cpp src/main1.cpp
cmd
Since src/func1.cpp is a prereq for all of the object files, they all get recompiled.
What you want instead is to use a static pattern rule:
obj/%.o : src/%.cpp
$(CC) $(CFLAGS) -I/src -c $< -o $#
Note that -I is for include directories, not include files.

Modify the makefile to include subdirectories

I'm working with a small program in c++ to learn the makefile.
The program has 2 source files (main.cpp and classf.cpp) and one header file (classf.h). All files are included in the project directory which is called "testmake". This is the generated makefile by eclipse on windlows:
CXXFLAGS = -O2 -g -Wall -fmessage-length=0
OBJS = main.o classf.o
LIBS =
TARGET = createPddl.exe
$(TARGET): $(OBJS)
$(CXX) -o $(TARGET) $(OBJS) $(LIBS)
all: $(TARGET)
clean:
rm -f $(OBJS) $(TARGET)
I would like to modify the makefile to accept new sub-directories, e.g, when I add a folder called "testmake/src" and move the file main.cpp inside it, folder called "testmake/csource" and move the classf.cpp inside it, and create a folder called "testmake/cheader" and move the classf.h inside it.
This makefile was genereated automatically by eclipse, and does not accept any changes. There for i have created manually a make file which is working with any c++ project that has a structure as tree.
I actually use this Makefile in general
CC := g++ # This is the main compiler
SRCDIR := src
BUILDDIR := build
TARGETDIR :=bin/
TARGET := pddlcrate
DATADIR := data
SRCEXT := cpp
SOURCES := $(shell find $(SRCDIR) -type f -name *.$(SRCEXT))
OBJECTS := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES:.$(SRCEXT)=.o))
CFLAGS := -g # -Wall
#LIB := -pthread -lmongoclient -L lib -lboost_thread-mt -lboost_filesystem-
mt -lboost_system-mt
INC := -I include
$(TARGET): $(OBJECTS)
#echo " Linking..."
#echo " $(CC) $^ -o $(TARGETDIR)$(TARGET) $(LIB)"; $(CC) $^ -o
$(TARGETDIR)$(TARGET) $(LIB)
$(BUILDDIR)/%.o: $(SRCDIR)/%.$(SRCEXT)
#mkdir -p $(BUILDDIR)
#echo " $(CC) $(CFLAGS) $(INC) -c -o $# $<"; $(CC) $(CFLAGS) $(INC) -c - o $# $<
clean:
#echo " Cleaning...";
#echo " $(RM) -r $(BUILDDIR) $(TARGET)"; $(RM) -r $(BUILDDIR) $(TARGET)
for any c++ project with this tree structure
$ tree .
├── Makefile
├── bin
>exefile
├── include
> *.h files
├── obj
> *.o files
├── src
>*.cpp

Why this makefile is always recompiling?

I've done a makefile for a project I'm working on andd my code is allways getting recompiled, and I don't know why... I've tried adding the .PHONY to the all target but didn't work. Here is the makefile:
TARGET = csv2sqlite
CXX = g++
CXXFLAGS = -std=c++11 -Wall -I $(INCDIR) -fdiagnostics-color=always -D_DEBUG
OBJECTS = csv.o main.o
BINDIR = bin
OBJDIR = obj
INCDIR = inc
SRCDIR = src
.PHONY:default
default: $(OBJECTS) $(TARGET)
%.o: $(SRCDIR)/%.cpp
#echo "Compiling dependecies..."
$(CXX) $(CXXFLAGS) -c $< -o $(OBJDIR)/$#
#echo "Done compiling $#"
$(TARGET): $(addprefix $(OBJDIR)/, $(OBJECTS))
#echo "Linking $# file..."
$(CXX) $(CXXFLAGS) $^ -o $(BINDIR)/$#
#echo "Linking $# file..."
run:
bin/$(TARGET)
clean:
rm -f obj/* bin/$(TARGET)
And the directory tree is:
project
├── inc
│ └── csv.hpp
├── src
│ ├── csv.cpp
│ └── main.cpp
├── obj
│ ├── csv.o
│ └── main.o
├── bin
│ └── csv2sqlite
└── makefile
Here is the complete project if you wanna see it. Also, what kind of improves I can do it?
Thanks!
Here:
%.o: $(SRCDIR)/%.cpp
...
$(CXX) $(CXXFLAGS) -c $< -o $(OBJDIR)/$#
#echo "Done compiling $#"
No, you're not. You didn't build foo.o, you built obj/foo.o, so the next time you tell Make to build foo.o, it will see that foo.o does not exist and attempt to rebuild it-- with this rule, which doesn't do what it claims to do. Likewise:
$(TARGET): $(addprefix $(OBJDIR)/, $(OBJECTS))
#echo "Linking $# file..."
$(CXX) $(CXXFLAGS) $^ -o $(BINDIR)/$#
#echo "Linking $# file..."
This doesn't build $(TARGET), it builds $(BINDIR)/$(TARGET), so Make runs this rule every time it's told to build $(TARGET).
The fix is simple:
OBJECTS = $(addprefix $(OBJDIR)/, csv.o main.o)
$(OBJDIR)/%.o: $(SRCDIR)/%.cpp
...
$(CXX) $(CXXFLAGS) -c $< -o $#
#echo "Done compiling $#"
TARGET = $(BINDIR)/csv2sqlite
$(TARGET): $(OBJECTS)
#echo "Linking $# file..."
$(CXX) $(CXXFLAGS) $^ -o $#
#echo "Linking $# file..."