patsubst in makefile doesnt replace the full filepath - c++

Im currently trying to get my makefile independend from my folder structure. So far i worked with a single specific folder containing all my .cpp files. My .o files are all placed in a seperate OBJ/ folder. However i now want my makefile to automatically grab all .cpp files from all subdirectories (any depth if possible) but still place all .o files the one same OBJ folder. This is how my makefile looks at the moment:
CC = g++ -g
SRC = Src/
OBJ = Obj/
BIN = Bin/
INC = -I Lib/GLFW/include/\
-I Lib/GLAD/include/\
-I Lib/STB/\
-I Lib/GLM/\
-I Src/Header/
LIB = -L Lib/GLFW/lib-mingw-w64 -lglfw3 -lopengl32 -lglu32 -lgdi32
EXE = $(BIN)Test.exe
SOURCEFILES = $(wildcard $(SRC)**/*.cpp)
OBJECTFILES = $(patsubst $(SRC)%.cpp,$(OBJ)%.o,$(SOURCEFILES))
all: $(SOURCEFILES) $(EXE)
$(EXE) : $(OBJECTFILES)
$(CC) -o $# $^ $(LIB)
$(OBJ)%.o: $(SRC)%.cpp
$(CC) -c $< $(INC) -o $#
It somewhat does what i want, but for some reason it currently is required to have the same folder structure of my SRC/ directory in OBJ/ aswell: If my .cpp file is SRC/foldername/name.cpp my makefile wants to create the .o as OBJ/foldername/name.o. I want it to be OBJ/name.o for all files regardless of the structure of SRC/ . What do i need to change to achieve that?
My second question is about the include path. As it can be seen in the makefile, all my header files are in a single directory Src/Header/. Is it possible to somehow also automatically get and link the right .h files from Src/ and it's subdirectories?

If you want to flatten paths, you can use notdir.
Unfortunately, because %.o is "magic", you can't just wrap it in a $(notdir).
By the way, as a matter of style, I'd leave the trailing slash out of those variables and use it in the expressions instead; it'll be much easier to read, and more flexible too.
This works for me:
SRC=srcPath
OBJ=objPath
SOURCEFILES = $(wildcard $(SRC)/**/*.cpp)
OBJECTFILES = $(addprefix $(OBJ)/, $(notdir $(patsubst $(SRC)/%.cpp, %.o, $(SOURCEFILES))))
all:
echo $(OBJECTFILES)
More pleasing alternatives surely exist, but the point is that you can use notdir to build your solution.

Flattening your object files in one single directory is not necessarily a very good idea. Your Makefile is getting more complex (see below) and there is a name collision possibility if, by accident, you have two source files with the same base name.
Anyway, it is what you want, and here is a possible solution:
SRC = Src
OBJ = Obj
SOURCEFILES = $(wildcard $(SRC)/**/*.cpp)
# $(1): full path to source file
define CompileRule
$$(OBJ)/$$(patsubst %.cpp,%.o,$$(notdir $(1))): $(1)
$$(CC) -c $$< $$(INC) -o $$#
endef
$(foreach f,$(SOURCEFILES),$(eval $(call CompileRule,$(f))))
It iteratively instantiates custom compilation rules for each of your source files. See the GNU make manual for a complete explanation.
I am not sure I fully understand your second question but if you want to find all header files in Src and build the list of -I<dir> compile options, you can probably try something like:
HEADERFILES := $(wildcard $(SRC)/**/*.h)
HEADERDIRS := $(sort $(dir $(HEADERFILES)))
INCLUDES := $(patsubst %,-I%,$(HEADERDIRS))
But it is very sub-optimal because every directory containing header files will be involved as -I<dir> in the compilation of every source file.
Last but not least, your next step could be "how do I automatically get the list of all header files a source file depends on?". If it is, you should probably have a look at this post about Auto-Dependency Generation.

Related

Makefile - No rule to make target `PATH/TO/file.o`

I'm really new to makefile and I'm struggling to make it compile all the .cpp files inside the src folder.
right now the makefile looks like this:
SRC_DIR = src
OBJ_DIR = build/obj
INCLUDE_DIR = $(SRC_DIR)/inc
SRC_DIRS = $(wildcard $(SRC_DIR) $(SRC_DIR)/* $(INCLUDE_DIR)/*)
C_FILES = $(filter %.cpp, $(wildcard $(SRC_DIRS)/*.cpp))
OBJ_FILES = $(addprefix $(OBJ_DIR)/, $(notdir $(C_FILES:.cpp=.o)))
INCLUDE_PATH = -I$(INCLUDE_DIR)
CXXFLAGS = $(INCLUDE_PATH)
CXX = g++
CFLAGS = -Wall -g
CC = gcc
$(OBJ_DIR)/%.o: $(SRC_DIRS)/%.cpp
pile: $(OBJ_FILES)
test:
#echo $(C_FILES)
#echo $(OBJ_FILES)
and what I want to achieve with each rule is:
pile: when calling make pile it should check for all the .o files inside build/obj.
$(OBJ_DIR)/%.cpp: find the equivalent .cpp file inside the source folder and compile it.
I suppose that there's something wrong with my second rule, but I can't really tell why cause the test rule is printing all the files.. :/
thank you in advance for any help.
Looks like SRC_DIRS could be an array which will not be happy with that matcher function.
Try changing:
$(OBJ_DIR)/%.o: $(SRC_DIRS)/%.cpp
to
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp
There are many problems here but the most fundamental is that you are not actually defining a rule when you write this line:
$(OBJ_DIR)/%.o: $(SRC_DIRS)/%.cpp
A rule consists of targets, prerequisites and a recipe. Here you have not provided a recipe, so you haven't told make how to actually build anything.
The other issue is as mentioned by #Ryan-Miller; you can't use multiple directories in a single rule like this. But you can just avoid the complexity anyway if all your source files are in a single directory.
If that's not what you want please make your question clear as to ALL the different things you need. You wrote "$(OBJ_DIR)/%.cpp: find the equivalente .cpp file inside the source folder and compiles it". How exactly are you defining "the equivalent .cpp file"? Computer programming is all about the details!

How to create a Makefile for a C++ project with multiple directories?

I want to create a Makefile for a project with the following layout:
Source files (.cpp, potentially .c) in /src, with potential subdirectories
Header files (.h, .hpp...) in /inc, with potential subdirectories
Object files (.o) in /obj, with potential subdirectories
External libraries in /lib
Compiled program in /bin
So far, I've managed to write together this Makefile, but with some issues:
SRC_DIR := src
BIN_DIR := bin
LIB_DIR := lib
INC_DIR := inc
OBJ_DIR := obj
SRCEXTS := .c .C .cc .cpp .CPP .c++ .cxx .cp
HDREXTS := .h .H .hh .hpp .HPP .h++ .hxx .hp
TARGETS := $(BIN_DIR)/program
SOURCES := $(wildcard $(addprefix $(SRC_DIR)/*,$(SRCEXTS)))
HEADERS := $(wildcard $(addprefix $(LIB_DIR)/*,$(HDREXTS)))
OBJECTS := $(addsuffix .o, $(basename $(SOURCES)))
CXX = g++
CXXFLAGS = -std=c++17 -c -g -Wall
.PHONY: all clean
all: $(TARGETS)
$(TARGETS): $(OBJECTS)
$(CXX) $^ -o $#
$(OBJ_DIR)%$(OBJECTS): $(SRC_DIR)%$(SOURCES)
$(CXX) $(CXXFLAGS) $< -o $#
clean:
rm -f $(OBJECTS) $(TARGETS)
I've tried to make it as "generic" as possible, so future projects could be started with this layout and makefile as a template. Currently, it creates the .o-files inside the src-directory alongisde the source code. It also fails when trying to compile the program with
g++ src/main.o -o bin/program
/usr/bin/ld: src/main.o: _ZSt4cout: invalid version 3 (max 0)
/usr/bin/ld: src/main.o: error adding symbols: bad value
collect2: error: ld returned 1 exit status
make: *** [Makefile:23: bin/program] Error 1
Very new to C++ development. Been on a wild goose-chase for a while, trying to get a clear image of how it all works. My code is basically a weird Frankenstein monster of several code snippets I've stumbled upon. Hopefully my intentions are clear enough, this is my last ditch effort! Thanks in advance :)
As #JohnBollinger points out, you are attempting too much at once. I will suggest a few changes to get your makefile off the ground.
I can't explain the error you get when you try to build the executable (you haven't given us enough information to reproduce the error), but it doesn't look like a Make problem. I suggest you try to build it without Make, using the command line, and see what happens.
I will assume that the names of your sources end in ".cpp" (such as src/sailboat/foo.cpp), the names of your headers end in ".hpp", and the directory tree under obj/ is already present and correct. These restrictions are temporary training wheels; you can remove them when you have more skill.
First, finding the source files. This:
SOURCES := $(wildcard $(addprefix $(SRC_DIR)/*,$(SRCEXTS)))
will not work if src/ has subdirectories. To recurse into subdirectories, we will use find. (There is a shortcut available to GNUMake, but for now we'll do things the slow and careful way).
SOURCES := $(shell find src -name "*.cpp")
Now to construct the names of the desired object files, such as obj/sailboat/foo.o. This:
OBJECTS := $(addsuffix .o, $(basename $(SOURCES)))
will give you src/sailboat/foo.o. We need a different command to change the leading directory as well as the suffix:
OBJECTS := $(patsubst src/%.cpp,obj/%.o,$(SOURCES))
Some of the source files refer to header files, so before we can start building objects, we must be able to supply them. The compiler can find the needed headers, but we must tell it where to search. So we need the directories, not the full paths:
HEADERS := $(shell find inc -name "*.hpp")
HEADERDIRS := $(sort $(dir $(HEADERS)))
(The sort is just to remove duplicates. Not necessary, but tidy.)
Now the rule to build the objects. This is incorrect:
$(OBJ_DIR)%$(OBJECTS): $(SRC_DIR)%$(SOURCES)
$(CXX) $(CXXFLAGS) $< -o $#
Remember that OBJECTS can contain several space-separated words. So if it contains foo bar, the target will be obj/%foo bar, which is clearly not what you intended. Likewise the prerequisite list is wrong, and the recipe too. Junk it and start over.
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp
$(CXX) $< -c -o $#
Then remember the header files, and add flags to tell the compiler where to look for them:
INCLUDEFLAGS := $(addprefix -I,$(HEADERDIRS))
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp
$(CXX) $< -c $(INCLUDEFLAGS) -o $#
That should be enough to get your makefile working; further refinements can wait.
I'm not mastering makefiles but one thing for sure : target "all" is definitively not a PHONY target because it often depends on other targets (in fact, it always depends on other targets, by definition !).
Also, as said John Bollinger, it is obviously possible to have only one makefile at the root.
"all" target is not mandatory : make will look for the first target fisrt.
The folder structure is like this :
----root
----src
----inc
----obj
----lib
Here's a simple one of mine (rules to build static libraries have been commented out) :
CC=g++
SRCDIR=./src/
INCDIR=./inc/
INCFLAG=-I$(INCDIR)
OBJDIR=./obj/
LIB=./lib/
LIBFLAG=-lstdc++
#-lmystaticlibrary -lmyclasses
#$(LIBFLAG) -L$(LIB)
#STATICLIBRARIES= mystaticlibrary myclasses
OBJECTS = $(OBJDIR)apprendre.o $(OBJDIR)myfunctions.o $(OBJDIR)myclasses.o
apprendre: $(OBJECTS)
# echo
# echo --------------------- Edition des liens
$(CC) $(OBJECTS) -o $# $(INCFLAG)
# mystaticlibrary: mystaticlibrary.o
# # echo
# # echo --------------------- Compilation librairie statique $#
# ar cr $(LIB)lib$#.a $(OBJ)$#.o
# ranlib $(LIB)lib$#.a
# myclasses: myclasses.o
# # echo
# # echo --------------------- Compilation librairie statique $#
# ar cr $(LIB)lib$#.a $(OBJ)$#.o
# ranlib $(LIB)lib$#.a
$(OBJDIR)%.o: $(SRCDIR)%.cpp $(INCDIR)myfunctions.h $(INCDIR)myclasses.h
# echo
# echo --------------------- Compilation $<
$(CC) -c $< $(INCFLAG) -o $#
run:
# echo
# echo "-----------------------> GO !!! : apprendre"
# ./apprendre

How to link objects in different directories in c++ with make without refering to them in the makefile?

I'm trying to clean up my project a little and I want to put object files and include files in a separate folder and be able to compile another makefile in my a different testing subdirectory. I would like to do this so that the make file in the testing directory doesn't have to know about the objects in the the above directory: I have been struggling all day trying to figure out make and compilation.
Not sure what I'm doing wrong but in addition to this question I would be appreciative of any information to straighten out my thinking about make and g++ so in the future I know where to look.
Anyways I have 2 Questions both with regards to my project layout:
Project
inc/
-- header files
-obj/
--object files
-source
-make file for project that compiles objects in obj directory (makefile0)
-testing/
--test1/
---test.cc
---makefile1
Question 1
So I want to include the header files in the inc directory in test.cc and then just focus on compiling test.cc like I would if I included a standard library header file. Right now I need to make reference to the object in ../../obj/ in makefile1 and would like to ignore that. and just do something simple like
g++ -I ../../inc/ -c test.cc
How is it possible to do this?
Question 2
In makefile0, for each source file I have to append a $(OBJ) or $(INC) to the front of any file I have in those folders and wondering if there is anyway to clean up my make file an do something like
Spinless2DFieldIndex.o: Spinless2DFieldIndex.cc Utils.o Dispersion.h
instead of
$(ODIR)/Spinless2DFieldIndex.o: Spinless2DFieldIndex.cc $(ODIR)/Utils.o $(INC)/Dispersion.h
The following should work:
Project/Makefile
objdir := obj/
vpath %.cc source
vpath %.o $(objdir)
CPPFLAGS := -Iinc -MMD -MP
.PHONY: all
all: testing/test1/test
include source/Makefile
include testing/test1/Makefile
Project/source/Makefile
override objects := $(objdir)obj.o
$(objects): $(objdir)%.o: %.cc
$(CXX) $(CXXFLAGS) $(CPPFLAGS) -c $(OUTPUT_OPTION) $<
clean:: ; $(RM) $(objects) $(objects:.o=.d)
-include $(objects:.o=.d)
Project/testing/test1/Makefile
override dir := $(dir $(lastword $(MAKEFILE_LIST)))
$(dir)test: obj.o
clean:: ; $(RM) $(dir)test $(dir)test.d
-include $(dir)test.d
This should allow for a certain amount of modularity, although the asymmetry in the makefiles betrays the fact that your idea of having a separate obj directory while at the same time wanting to have the test executables in their own directory is perhaps not the best way to organize things. Personally I use a more configure style of makefile that recreates the project tree in the current working directory which helps separate the source from the build.

Make and c++ files, "includes" in the cpp

This is my current makefile:
# Compiler #
CXX = g++
DEBUG = -g
LFLAGS =
CXXFLAGS = -Wall
# Directories #
SRCDIR = src/
INCDIR = include/
BUILDDIR = build/
BINDIR = bin/
# Objects #
OBJ_NAMES = main.o dfa.o dfaException.o state.o
OBJS = $(addprefix $(BUILDDIR), $(OBJ_NAMES))
# Output #
NAME = pract3
TARGET = $(BINDIR)pract3
# Clean #
ifeq ($(OS),Windows_NT)
RM = del /q /s $(BUILDDIR:/=\)*.o $(BINDIR:/=\)$(NAME)*
else
RM = rm -rf $(BUILDDIR)*.o $(TARGET)*
endif
# Files #
$(TARGET): $(OBJS)
$(CXX) $(CXXFLAGS) $(LFLAGS) $(OBJS) -o $(TARGET)
$(BUILDDIR)%.o: $(SRCDIR)%.cpp
$(CXX) $(CXXFLAGS) $(LFLAGS) -c $< -o $#
clean:
$(RM)
And this is my project tree:
Porject/
bin/
build/
doc/
...
include/
connection.hpp
dfa.hpp
dfaException.hpp
state.hpp
src/
dfa.cpp
dfaException.cpp
main.cpp
state.cpp
Makefile
Readme
Now, I'm having three "problems".
First I want my makefile to create the bin and build directories just in case they aren't. I guess I just have to use:
mkdir $(BUILDDIR)
mkdir $(BINDIR)
But where should I put them ? And also, how can I prevent mkdir and rm (or del in windows) messages like "Cant find ..." or "x directory already exists ... "
Second, I'm guessing I can read the objects name from src/ (converting the .cpp to .o), but how can I read the file names from a directory ?
Last, I have one template class: connection.hpp (all functions are in the header). This file is included in state.hpp using: #include "../include/connection.hpp". I ran make one time with all correct and then I purposely make a syntax error in connection.hpp. Then I ran make again, but It only compiled the target file using the .o files in build without any error. Everytime I want to edit connection.hpp I have to use make clean and then make. Is there any better way to do this ?
If you need a directory to exist before you can proceed, simply put
mkdir -p ${DIRECTORY}
before you need it in your rule. If the directory already exists, mkdir -p will happily do nothing.
Likewise, if you use rm -f FILE, it should not complain if FILE does not exist in the first place.
There is no portable way to create a variable that holds the names of all files in a directory. But you are already using GNU Make features anyway, so you can just use
SOURCES = $(wildcard ${SRCDIR}/*.cpp)
and then
OBJECTS = $(SOURCES:.cpp=.o)
to transform them into object file names. You'll probably want to replace the leading directory name as well, I guess.
You didn't list any *.hpp files as prerequisites in your make file. You can either manually add them like
foo.o: foo.cpp bar.hpp baz.hpp
but that becomes unpleasant very quickly. Another trick is to use the compiler to tell you the headers (transitively) #included by a file. If you are using GCC, you can run
gcc -MM foo.cpp
to have it output above make-file snippet. You can put a pattern rule like the following
%.deps: %.cpp
${CXX} -MM ${CPPFLAGS} $< > $#
into your make-file and then include the generated *.deps files.
include $(SOURCES:.cpp=.deps)
GNU Make will be smart enough to first parse the make-file, recognize that the *.deps files don't exist and therefore cannot be included but figure out that there is a rule to generate them. So it will execute that rule and then continue parsing the make-file.
I have learned that trick from Peter Miller's great article Recursive Make Considered Harmful which is a good read if you want to learn how to write good make-files.

How to get cpp files from different directories compiled into one folder?

I have a number of C++ files distributed in several folders.
a_library/
file1.cpp
file2.cpp
category1/
file3.cpp
file4.cpp
They are guaruanteed to be uniquely named. I want to compile all those C++ files to seperate Object-files in the obj/ directory.
I have a list of all source-files with relative path, and their corresponding destination-names.
a_library/file1.cpp
a_library/file2.cpp
a_library/category1/file3.cpp
a_library/category1/file4.cpp
obj/file1.obj
obj/file2.obj
obj/file3.obj
obj/file4.obj
How can I make a rule that will convert a C++ file from the first list to a object-file from the second one?
These attempts do not work:
obj/%.obj: %:cpp
# ...
%.obj: %.cpp
# ...
.cpp.obj:
# ...
I would like to not write rules like this:
obj/%.obj: a_library/%.cpp
# ...
obj/%.obj: a_library/category1/%.cpp
# ...
Try setting VPATH:
VPATH = a_library:a_library/category1
obj/%.o: %.cpp
$(CXX) -c $(CPPFLAGS) $(CFLAGS) $(CXXFLAGS) -o $# $<
And to add complete file list (I would recommend you explicitely list the files, do not use $(wildcard ...) function) and linking of the application:
files := main.cpp $(wildcard a_library/*.cpp) a_library/category1/file.cpp
obj/application: $(patsubst %.cpp,obj/%.o,$(notdir $(files)))
$(CXX) $(CFLAGS) $(CXXFLAGS) $(LDFLAGS) -o $# $+
The $(wildcard) has an annoying tendency to pick up anything in the directories, like one-off test files or temporaries (if they happen to have a fitting name: ~file.cpp).
One solution I can think of: just build them inplace with a simple rule and then make a "collection phase" moving the ".o" files to a single folder.
Make a "collect_objs" target which depends on your $(OBJS) and then your "main" target must depend on "collect_objs".
The traversal can be done using shell
dirs := $(shell find ./ -type d)
collect_objs: $(dirs)
for d in $+; do \
mv *.o YourDestDir/*.o
done
Of course, this implies using UnxUtils package (with 'find' and 'mv') or Cygwin since you are on Windows.
The other option is to generate the targets for each of your .c/.cpp file explicitly, using some tool. Grab python, traverse source directories and for each .c/.cpp file write
obj/file_name.o:
gcc -c path/fo/file_name.c -o obj/file_name.o
Use cmake to make the build configuration for you.
Some time ago I set up a simple example project on github.
The standard way is to have a Makefile in each folder and call recursively with include
This was my first two hits on 10^100:
http://owen.sj.ca.us/~rk/howto/slides/make/slides/makerecurs.html
http://www.gnu.org/savannah-checkouts/gnu/make/manual/html_node/Recursion.html
Not strictly related to this question as it doesn't have to do with Make, though I'd like to show how I compile my projects now, 3 years later. Craftr is a Python based meta build system that encourages indirect out-of-tree builds (eg. a build in the working tree). Building object files and cresting a static library is as easy as
# craftr_module(my_project)
from craftr import *
from craftr.ext.platform import cxx, ar
objects = cxx.compile(
sources = path.platform('**/*.cpp'),
)
lib = ar.staticlib(
output = 'myproj',
inputs = [objects],
)
Running craftr -eb will result with the following structure of build products
Craftfile
file1.c
file2.c
category1/
file3.c
file4.c
build/
my_project/
obj/
file1.o
file2.o
category1/
file3.o
file4.o
libmyproj.a