Partially compile and include boost with makefile (c++) - c++

I have a project in c++ that I want to be reasonably portable, and can be compiled by issuing a single "make" command. I currently use boost headers for certain operations, but now need "filesystem" from boost, which requires compilation.
I know there is a way to compile boost using the included shell script, but that takes FOREVER. I am curious of a way to selectively compile boost libraries, as specified from a makefile and include them in the linking process.
My current general idea is:
Have a makefile variable of the boost libraries wanted to be compiled and included.
Compile those libraries (before any dependent projects) from a boost distribution folder, AND output compiled binaries to a specific folder in the makefile project (eg: a "lib" folder)
I understand how to have these files included in the project by using:
-Llib
What is the best way to do this, as well as choose where the boost output from compilation is? Should the files be .a or .o files? (or no extension?)

This is the wilder older brother of the prior answer's manual makefile. Given a path to the BOOST distro, BOOST_DISTRO, it finds the relevant source files for the target libs specified in TARGET_BOOST_LIBS, compiles them into $(WORK_FOLDER)/<lib>/, and archives the resulting objects into $(DEST_FOLDER)/libboost_<lib>.a.
BOOST_DISTRO=.
DEST_FOLDER=libs
WORK_FOLDER=build
TARGET_BOOST_LIBS=\
system \
filesystem \
serialization
.PHONY: all
all: $(foreach lib,$(TARGET_BOOST_LIBS),$(DEST_FOLDER)/libboost_$(lib).a )
$(DEST_FOLDER):
mkdir -p $(DEST_FOLDER)
$(WORK_FOLDER):
mkdir -p $(WORK_FOLDER)
#####
# helper for building the .o files in WORK_FOLDER
#####
define MAKE_BOOST_LIB_COMPILE_RULES
$(foreach cppfile,$(shell ls $(BOOST_DISTRO)/boost/libs/$(1)/src/*.cpp),$(WORK_FOLDER)/$(1)/$(notdir $(cppfile:.cpp=.o)): $(cppfile) | $(WORK_FOLDER)/$(1)
$(CXX) $(CXXFLAGS) -D BOOST_ALL_NO_LIB \
-I$(BOOST_DISTRO)/boost \
-c $$^ \
-o $$#
)
endef
#####
# define the build rules based on the files we find in the subfolders of
# the boost distro that correspond to our library names
#####
define BUILD_BOOST_LIB
$(WORK_FOLDER)/$(1): | $(WORK_FOLDER)
mkdir -p $$#
$(call MAKE_BOOST_LIB_COMPILE_RULES,$(1))
$(DEST_FOLDER)/libboost_$(1).a: $(foreach cppfile,$(notdir $(shell ls $(BOOST_DISTRO)/boost/libs/$(1)/src/*.cpp)),$(WORK_FOLDER)/$(1)/$(cppfile:.cpp=.o)) | $(DEST_FOLDER)
ar r $$# $$^
ranlib $$#
endef
#####
# dynamically generate the build rules from the list of libs
#####
$(foreach lib,$(TARGET_BOOST_LIBS),$(eval $(call BUILD_BOOST_LIB,$(lib))))
.PHONY: clean
clean:
-rm -rf $(WORK_FOLDER)
-rm -rf $(DEST_FOLDER)
Testing with my ancient BOOST (#define BOOST_VERSION 105500), this builds the listed libs, and a dummy test program successfully compiles and calls boost::filesystem::absolute().

Using lockcmpxchg8b's answer, I was able to adapt it to my needs.
It was a little tricky to get makefiles to work right when recursively called, but I was able to get it to work.
SHELL = /bin/sh
# This makefile expects multiple arguments to be passed:
#
# Use the pattern: make var_name="var_value" when invoking this makefile
#
# BOOST_VER (the version suffix )
# BOOST_LIBS_TO_BUILD (space delimited list of boost libraries to build)
# BOOST_LIB_DIR (the output lib dir for the boost libraries)
# BOOSTDIR (the base directory to build from)
#
# Compile Info
CXX = g++
CXXFLAGS = -Wall -std=c++11
WORK_FOLDER = obj_boost$(BOOST_VER)
.PHONY: all
all: $(foreach lib,$(BOOST_LIBS_TO_BUILD),$(BOOST_LIB_DIR)/libboost_$(lib).a )
$(BOOST_LIB_DIR):
#mkdir -p $(BOOST_LIB_DIR)
$(WORK_FOLDER):
#mkdir -p $(WORK_FOLDER)
#####
# helper for building the .o files in WORK_FOLDER
#####
define MAKE_BOOST_LIB_COMPILE_RULES
$(foreach cppfile,$(shell ls $(BOOSTDIR)/libs/$(1)/src/*.cpp),$(WORK_FOLDER)/$(1)/$(notdir $(cppfile:.cpp=.o)): $(cppfile) | $(WORK_FOLDER)/$(1)
$(CXX) $(CXXFLAGS) -D BOOST_ALL_NO_LIB \
-I$(BOOSTDIR) \
-c $$^ \
-o $$#
)
endef
#####
# define the build rules based on the files we find in the subfolders of
# the boost distro that correspond to our library names
#####
define BUILD_BOOST_LIB
$(WORK_FOLDER)/$(1): | $(WORK_FOLDER)
#mkdir -p $$#
$(call MAKE_BOOST_LIB_COMPILE_RULES,$(1))
$(BOOST_LIB_DIR)/libboost_$(1).a: $(foreach cppfile,$(notdir $(shell ls $(BOOSTDIR)/libs/$(1)/src/*.cpp)),$(WORK_FOLDER)/$(1)/$(cppfile:.cpp=.o)) | $(BOOST_LIB_DIR)
#ar r $$# $$^
#ranlib $$#
endef
#####
# dynamically generate the build rules from the list of libs
#####
$(foreach lib,$(BOOST_LIBS_TO_BUILD),$(eval $(call BUILD_BOOST_LIB,$(lib))))
.PHONY: clean
clean:
#rm -rf $(WORK_FOLDER)
#rm -rf $(BOOST_LIB_DIR)/*
#echo "---- Done Cleaning Boost Libs ----"
To call this makefile, I call it from another makefile using the following:
# Boost
BOOST_VER= _1_65_1
BOOST_LIBS_TO_BUILD = filesystem timer chrono
#relative to this file
BOOST_LIB_DIR = shared/lib_boost$(BOOST_VER)
#relative to this file
BOOSTDIR = shared/boost$(BOOST_VER)
#Subdirectories
DIRECTORIES = $(sort $(dir $(wildcard */makefile)))
.PHONY: build
build: dependencies
#$(foreach dir,$(DIRECTORIES),$(MAKE) -C $(dir);)
dependencies:
#echo "---- Build Dependencies ----"
#$(MAKE) -C shared -f build_boost_libs.mk BOOST_LIBS_TO_BUILD="$(BOOST_LIBS_TO_BUILD)" BOOST_LIB_DIR="../$(BOOST_LIB_DIR)" BOOSTDIR="../$(BOOSTDIR)" BOOST_VER="$(BOOST_VER)"
The pattern to CORRECTLY invoke a recursive make is:
$(MAKE) -C subdir_of_makefile -f makefile_name.mk
The use of $(MAKE) is the proper way to invoke make (as per documentation -- it ensures the same make command is used as the top-most make) and to correctly change the running directory, you need to use the -C argument (otherwise the directory context of any sub-make is the top-most parent makefile in the call stack). The -f option specifies a makefile that is anything but a default name for a makefile. This takes into account the -C option when looking for the named makefile.
I also found out that passing "arguments" to a makefile is a bit tricky, and is usually done with export and unexport, but the problem with those is due to ONLY the LAST "status" of a variable to be exported is used for the ENTIRE makefile. You can't export a variable, call a sub-make, then unexport. It would be unexported for the entire makefile run (as the last "status" of the variable was to be unexported despite it being called after the sub-make). Variable "status" is computed/parsed before the makefile executes.
Thanks for the help! I appreciate it. I'll also be releasing this makefile in an open-source project in the future (I'll credit your username as helping with that)

Related

Makefile working in Linux but not under Windows, can't find files in subdirectory

I have a makefile (gnu make) working under linux, but when porting it to windows it does not work. The goal of the makefile is to make all the *.cpp file that reside in different subdirectories and compile them into *.obj files in a single BUILD_DIR.
Between linux and windows I only adjusted the SOURCES variable, the linux line is shown but commented. When I check all the names and directories they show the same (relative paths in their perspective notation) and what I expect.
The "error message" I get on windows is: make: *** No rule to make target 'DEM.cpp', needed by 'DEM.obj'. Stop.
In debug mode it says: File 'DEM.cpp' does not exist. (which it obviously does)
On linux it finds the file via the VPATH, in debug mode it says: No need to remake target 'DEM.cpp'; using VPATH name './Code/DEM.cpp'.
The subdirectory structure in both linux and windows is the same and the makefile is run from the same location.
Question: What is going wrong here, I have the feeling it has something to do with VPATH being handled differently on windows but I am not certain.
My makefile is:
# MAKEFILE
# Some makefile settings
SHELL = /bin/sh #Make sure the shell is the standard shell
.SUFFIXES: #Clear al implicit suffixes
.SUFFIXES: .cpp .obj #Set used suffixes
# Variable Declaration
CXX := g++
BUILD_DIR = .\bin\Release
PROGRAM_NAME := DEM.exe
#SOURCES := $(shell find . -name '*.cpp') #All *.cpp files with directory (LINUX style)
SOURCES := $(shell FORFILES /S /M *.cpp /C "CMD /C ECHO #relpath") #All *.cpp files with directory
NAMES := $(notdir $(basename $(SOURCES))) #Get all files names of the *.cpp files without extensions
OBJECTS := $(addsuffix .obj, $(NAMES)) #Get the to be generate *.o files without directory
SRC_DIRS := $(dir $(SOURCES)) #All the directory in which the sources are found
VPATH := $(SRC_DIRS) $(BUILD_DIR)
.PHONY: all
all: build #Standard entry point, run release
#echo "BUILD DONE"
.PHONY: build
build: $(PROGRAM_NAME)
#echo "BUILD DONE"
$(PROGRAM_NAME): $(OBJECTS)
#echo "Link executable"
$(CXX) -o $(BUILD_DIR)/$(PROGRAM_NAME) $(addprefix $(BUILD_DIR)/,$(OBJECTS))
$(OBJECTS): %.obj: %.cpp
#echo "Compile into object code: $<"
${CXX} $(CXXFLAGS) -c $< -o $(BUILD_DIR)/$# #Compiles object code and places it in the $(BUILD_DIR)
Update 1: Based on the comment of some programmer dude, I ran it with -p, and got the following interesting result:
# Not a target:
DEM.cpp:
# Implicit rule search has been done.
# File does not exist.
# File has not been updated.
# Not a target:
World.cpp:
# Implicit rule search has not been done.
# Modification time never checked.
# File has not been updated.
# Lot more targets below
It seems only to not find DEM.cpp but it finds everything else.
DEM.cpp resides in C:\Users\dklomp\Documents\Programming\C++ Source\DEM\DEM\Code, but should resolve to .\Code\
Most other files reside in subdirectories in Code.
But stdafx.cpp also resides in .\Code\ and there is no problem finding it.
Can it be a name clash with the directory name DEM being similar to DEM.cpp
Update 2: For information and closure.
I had already check the VPATH by reading the variable by printing it this seemed to give the correct information. However if i read it with #(info $(VPATH)) it seemet empty:
#echo "VPATH print variable: "
#echo "$(VPATH)"
#echo "VPATH print info: "
$(info VPATH)
gave:
"VPATH print variable: "
"./Code/InputOutput/ ./Code/InputOutput/ ./Code/InputOutput/
./Code/InputOutput/ ./Code/ ./Code/ ./Code/Models/ ./Code/Models/
./Code/Forces/ ./Code/Forces/ ./Code/Forces/ ./Code/Forces/ ./Code/Forces/
./Code/Forces/ ./Code/Forces/ ./Code/Forces/ ./Code/Forces/ ./Code/Tools/
./Code/Tools/ ./Code/Tools/ ./Code/Tools/ ./Code/Solvers/ ./Code/Solvers/
./Code/Solvers/ ./Code/World/ ./Code/World/ ./Code/World/ ./Code/World/
./Code/World/ ./Code/Interactions/ ./Code/Interactions/ ./Code/Interactions/
./Code/Interactions/ ./Code/Interactions/ ./Code/Interactions/
bin\Release"
"VPATH print info: "
Indeed with a lot of repetitions (will use the sort suggestion of hardcoreHenry) but it seemed oke. However the VPATH info is empty for some reason.
However implementing the proposed solution by code_fodder and removing all the inline code comments works. The inline code comments on the variable declaration do not seem to matter, those in the rules section do for windows, linux seems to handle inline code comments fine everywhere.
As usually thanks for the help and suggestions.
I tested this in a path that had spaces before the current location (e.g. like d:\path with spaces\myfolder\sub_folder1\sub_folder2\).
I think there is some issue with the FORFILES syntax/output - at least I could not get it to work. As soon as I used the more generic rwildcard from second answer from here it started to work quite well. Then I added rules to create/clean your output folder.
Also I removed any inline comments, since I am not 100% sure they are valid because they leave space characters everywhere and makefile is not very tolerant of that... I.E. make-sure you don't leave any trailing white space on any lines.
Also - this all assumes its gnu-make and not nmake
# MAKEFILE
# Some makefile settings
SHELL = /bin/sh
.SUFFIXES:
.SUFFIXES: .cpp .obj
# Variable Declaration
CXX := g++
BUILD_DIR = .\bin\Release
PROGRAM_NAME := DEM.exe
# Define a recursive wild card function that is portable.
rwildcard=$(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d))
#SOURCES := $(shell FORFILES /S /M *.cpp /C "CMD /C ECHO #relpath")
SOURCES := $(call rwildcard, ./, *.cpp)
$(info SOURCES: $(SOURCES))
NAMES := $(notdir $(basename $(SOURCES)))
OBJECTS := $(addsuffix .obj, $(NAMES))
SRC_DIRS := $(dir $(SOURCES))
VPATH := $(SRC_DIRS) $(BUILD_DIR)
.PHONY: all
all: create_dirs build
#echo "BUILD DONE"
# Has an order dependency on create_dirs to ensure that happens first, and so
# it works with parallel build (e.g. `make -j4`)
.PHONY: build
build: $(PROGRAM_NAME) | create_dirs
#echo "BUILD DONE"
$(PROGRAM_NAME): $(OBJECTS)
#echo "Link executable"
$(CXX) -o $(BUILD_DIR)/$(PROGRAM_NAME) $(addprefix $(BUILD_DIR)/,$(OBJECTS))
$(OBJECTS): %.obj: %.cpp
#echo "Compile into object code: $<"
${CXX} $(CXXFLAGS) -c $< -o $(BUILD_DIR)/$#
.PHONY: create_dirs
create_dirs:
#echo "creating dirs"
mkdir -p $(BUILD_DIR)
.PHONY: clean
clean:
#echo "cleaning"
rm -rf $(BUILD_DIR)

Make and c++ files, "includes" in the cpp

This is my current makefile:
# Compiler #
CXX = g++
DEBUG = -g
LFLAGS =
CXXFLAGS = -Wall
# Directories #
SRCDIR = src/
INCDIR = include/
BUILDDIR = build/
BINDIR = bin/
# Objects #
OBJ_NAMES = main.o dfa.o dfaException.o state.o
OBJS = $(addprefix $(BUILDDIR), $(OBJ_NAMES))
# Output #
NAME = pract3
TARGET = $(BINDIR)pract3
# Clean #
ifeq ($(OS),Windows_NT)
RM = del /q /s $(BUILDDIR:/=\)*.o $(BINDIR:/=\)$(NAME)*
else
RM = rm -rf $(BUILDDIR)*.o $(TARGET)*
endif
# Files #
$(TARGET): $(OBJS)
$(CXX) $(CXXFLAGS) $(LFLAGS) $(OBJS) -o $(TARGET)
$(BUILDDIR)%.o: $(SRCDIR)%.cpp
$(CXX) $(CXXFLAGS) $(LFLAGS) -c $< -o $#
clean:
$(RM)
And this is my project tree:
Porject/
bin/
build/
doc/
...
include/
connection.hpp
dfa.hpp
dfaException.hpp
state.hpp
src/
dfa.cpp
dfaException.cpp
main.cpp
state.cpp
Makefile
Readme
Now, I'm having three "problems".
First I want my makefile to create the bin and build directories just in case they aren't. I guess I just have to use:
mkdir $(BUILDDIR)
mkdir $(BINDIR)
But where should I put them ? And also, how can I prevent mkdir and rm (or del in windows) messages like "Cant find ..." or "x directory already exists ... "
Second, I'm guessing I can read the objects name from src/ (converting the .cpp to .o), but how can I read the file names from a directory ?
Last, I have one template class: connection.hpp (all functions are in the header). This file is included in state.hpp using: #include "../include/connection.hpp". I ran make one time with all correct and then I purposely make a syntax error in connection.hpp. Then I ran make again, but It only compiled the target file using the .o files in build without any error. Everytime I want to edit connection.hpp I have to use make clean and then make. Is there any better way to do this ?
If you need a directory to exist before you can proceed, simply put
mkdir -p ${DIRECTORY}
before you need it in your rule. If the directory already exists, mkdir -p will happily do nothing.
Likewise, if you use rm -f FILE, it should not complain if FILE does not exist in the first place.
There is no portable way to create a variable that holds the names of all files in a directory. But you are already using GNU Make features anyway, so you can just use
SOURCES = $(wildcard ${SRCDIR}/*.cpp)
and then
OBJECTS = $(SOURCES:.cpp=.o)
to transform them into object file names. You'll probably want to replace the leading directory name as well, I guess.
You didn't list any *.hpp files as prerequisites in your make file. You can either manually add them like
foo.o: foo.cpp bar.hpp baz.hpp
but that becomes unpleasant very quickly. Another trick is to use the compiler to tell you the headers (transitively) #included by a file. If you are using GCC, you can run
gcc -MM foo.cpp
to have it output above make-file snippet. You can put a pattern rule like the following
%.deps: %.cpp
${CXX} -MM ${CPPFLAGS} $< > $#
into your make-file and then include the generated *.deps files.
include $(SOURCES:.cpp=.deps)
GNU Make will be smart enough to first parse the make-file, recognize that the *.deps files don't exist and therefore cannot be included but figure out that there is a rule to generate them. So it will execute that rule and then continue parsing the make-file.
I have learned that trick from Peter Miller's great article Recursive Make Considered Harmful which is a good read if you want to learn how to write good make-files.

Unable to find headers with make

I have a (fairly simple) makefile adapted from here that I am attempting to use to build a project on Ubuntu. The project tree is fairly simple: Makefile is in the root project directory, and there are src/, include/, build/, and bin/, where source code, header files, object files, and executables are stored, respectively.
When I run make from the root directory of the project, I get the following error message:
Linking...
g++ src/Main.cpp src/Foo.cpp -o bin/runner
src/Main.cpp:1:19: fatal error: Foo.hpp: No such file or directory
#include "Foo.hpp"
^
compilation terminated.
src/Foo.cpp:1:19: fatal error: Foo.hpp: No such file or directory
#include "Foo.hpp"
^
compilation terminated.
make: *** [bin/runner] Error 1
All that's currently in the project is Main.cpp. which calls two test functions Foo() and Bar() from Foo.cpp, which references a header file Foo.hpp. Here is the makefile:
CC := g++ # This is the main compiler
SRCDIR := src # Directory for source code
BUILDDIR := build # Directory containing all object files, which are removed on "make clean"
TARGET := bin/runner # bin/runner contains the main executable for project
# bin/ contains all other executables in the project (such as tests)
SRCEXT := cpp # File extension of source code
# Look for all the source files in SRCDIR with the file extension specified above
SOURCES := $(shell find $(SRCDIR) -type f -name *.$(SRCEXT))
# Name all object files the same root name as the source files from which they came, but add a .o extension to the end
OBJECTS := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES:.$(SRCEXT)=.o))
# The -g flag specifies that debugging information should be produced in the native format of the OS
CFLAGS := -g -Wall
# Various flags for libraries that might need to be linked
INC := -I include # Ensures that all header files (in the include/ folder) are accessible for build
# Show the components that are currently being compiled/linked
# Also, this is the main procedure for make: The TARGET is built from the objects, and
# object files are built from source
$(TARGET): $(OBJECTS)
#echo " Linking..."
#echo " $(CC) $^ -o $(TARGET)"; $(CC) $^ -o $(TARGET)
$(BUILDDIR)/%.o: $(SRCDIR)/%.$(SRCEXT)
#mkdir -p $(BUILDDIR)
#echo " $(CC) $(CFLAGS) $(INC) -c -o $# $<"; $(CC) $(CFLAGS) $(INC) -c -o $# $<
# Directives for "make clean" which cleans all object files out of the build/ folder
clean:
#echo " Cleaning...";
#echo " $(RM) -r $(BUILDDIR) $(TARGET)"; $(RM) -r $(BUILDDIR) $(TARGET)
# Destroys everything in the build/ and bin/runner/ folders. Does not clean test executables.
.PHONY: clean
What am I missing here in order to get the header files to be properly linked?
EDIT: Here is the new makefile, and the current output:
# This is the main compiler
CC := g++
# Directory for source code
SRCDIR := src
# Directory containing all object files, which are removed on "make clean"
BUILDDIR := build
# bin/runner contains the main executable for project
# bin/ contains all other executables in the project (such as tests)
TARGET := bin/runner
# File extension of source code
SRCEXT := cpp
# Ensures that all header files (in the include/ folder) are accessible for build
INC := -I/include
# Look for all the source files in SRCDIR with the file extension specified above
# SOURCES := $(shell find $(SRCDIR) -type f -name *.$(SRCEXT))
SOURCES := $(wildcard $(SRCDIR)/*.$(SRCEXT))
# Name all object files the same root name as the source files from which they came, but add a .o extension to the end
# OBJECTS := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES:.$(SRCEXT)=.o))
OBJECTS := $(addprefix $(TARGET)/, $(notdir $(SOURCES:.cpp=.o)))
# The -g flag specifies that debugging information should be produced in the native format of the OS
CFLAGS := -g -Wall
# Various flags for libraries that might need to be linked
LIB := #-pthread -lmongoclient -L lib -lboost_thread-mt -lboost_filesystem-mt -lboost_system-mt
# Show the components that are currently being compiled/linked
# Also, this is the main procedure for make: The TARGET is built from the objects, and
# object files are built from source
$(TARGET): $(OBJECTS)
#echo " Linking..."
$(CC) $^ -o $(TARGET)
# #echo " $(CC) $^ -o $(TARGET) $(LIB)"; $(CC) $^ -o $(TARGET) $(LIB)
$(BUILDDIR)/%.o: $(SRCDIR)/%.$(SRCEXT)
#mkdir -p $(BUILDDIR)
$(CC) $(CFLAGS) $(INC) -c -o $# $<
# Directives for "make clean" which cleans all object files out of the build/ folder
clean:
#echo " Cleaning...";
#echo " $(RM) -r $(BUILDDIR) $(TARGET)"; $(RM) -r $(BUILDDIR) $(TARGET)
# Tests
# tester:
# $(CC) $(CFLAGS) test/tester.cpp $(INC) $(LIB) -o bin/tester
# Spikes
# ticket:
# $(CC) $(CFLAGS) spikes/ticket.cpp $(INC) $(LIB) -o bin/ticket
# Destroys everything in the build/ and bin/runner/ folders. Does not clean test executables.
.PHONY: clean
Output:
[scott]> make
make: *** No rule to make target `bin/runner/Foo.o', needed by `bin/runner'. Stop.
tl;dr
Don't put end-of-line comments on variable assignments in make. It doesn't work the way you might expect.
Explanation
Your makefile isn't running the steps you expect it is.
You shouldn't be seeing Linking... for the compilation step.
make shouldn't be attempting to create the target from the source .cpp files.
You should be seeing your INC and CFLAGS values on the compilation line (but you are getting linking output so obviously aren't seeing them).
That's why your header can't be found by the way, your linking line doesn't have -I on it anywhere.
The reason that's happening is because make is applying the wrong rule.
make is applying the wrong rule because your variables are being set incorrectly.
Your variables are being set incorrectly because your variables have values you don't expect.
The makefile you started from had errors the author wasn't aware of.
make is not always very smart.
When you write
FOO := some value # comment
you expect FOO to have the value some value but make sees things differently.
make gives it the value some value since it can't tell the difference between the space between some and value and the space after value and before the comment.
So when you run your shell command (with *.$(SRCEXT) unquoted) the shell just ignores the trailing spaces). (Try quoting *.'$(SRCEXT)' and see what you get.)
However when you then try to $(SOURCES:=.$(SRCEXT)=.o) make doesn't drop the spaces and you have actually written $(src/Main.cpp src/Foo.cpp:=cpp =.o) which, you may notice, is a pattern that doesn't actually match.
As a result $(OBJECTS) gets the unmodified value of $(SOURCES) and "confuses" the $(TARGET): $(OBJECTS) line later causing make to skip your compilation target.
(Oh, also, that's why your linking line has a million spaces between g++ and the first source file.)
Oh, also, you don't need to shell out for find there unless your src/ directory has sub-directories of its own (and even then not with some make magic) because $(wildcard $(SRCDIR)/*.$(SRCEXT)) will work just fine (and also would have failed earlier I believe given this problem).
Define an environment variable that has . (current working directory) first, then ./include (not just include subdirectory but as ,/include and rest of the INCLUDE dirs that you might already have because of the compiler or other software requirement)
set INCLUDE :=.:./include:$INCLUDE
Alternately, use:
INC := -I. -I./include
gcc -I option is as: -I dir
Adds the directory dir to the list of directories to be searched for header files. Directories named by '-I' are searched before the standard system include directories. If the directory dir is a standard system include directory, the option is ignored to ensure that the default search order for system directories and the special treatment of system headers are not defeated

Makefile target always up to date with code generation

I have a script which generates multiple C++ .h and .cpp files, based on a configuration file. This script also generates a file called 'Makefile.inc', and this file contains a variable with the required object filenames, for the generated .cpp files.
Example of a Makefile.inc file (all paths are absolute):
MESSAGE_OBJS = \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/error-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/challenge-request-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/challenge-response-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/login-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/get-game-list-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/game-list-response-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/join-game-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/connect-to-game-message.o \
/scratch/openttd/software/AtLargePlatform/branches/lucas/libatlarge/atlarge/messages/leave-game-message.o
Using the answer in this question as base, I created the following Makefile:
# Include the generated makefile for messages.
# This includes a variable with all message targets
include atlarge/messages/Makefile.inc
# Create a variable with all source targets
LIBOBJS = \
atlarge/exceptions.o \
atlarge/message-factory.o \
atlarge/envelope.o \
atlarge/client.o \
atlarge/user.o \
atlarge/atlarge-protocol.o \
atlarge/atlarge-gameserver.o \
$(MESSAGE_OBJS)
CXXFLAGS += -W -Wall -I. -g -O3 -MD \
`pkg-config jansson --cflags` \
`libgcrypt-config --cflags` \
`pkg-config glib-2.0 --cflags` \
-fPIC -DDEBUG -DENABLE_LOGGING
PREFIX = /usr/local
# TODO use pkg-config for jansson
LDLIBS += -lm -ljansson -latlarge-util `libgcrypt-config --libs` `pkg-config glib-2.0 --libs`
LDFLAGS += -shared -L/usr/local/lib
# Include automatically generated dependencies
-include $(LIBOBJS:.o=.d)
all: libatlarge.so
# If the message Makefile doesn't exist yet, generate it
atlarge/messages/Makefile.inc: atlarge/messages/messages.conf
python ../common/messagegen.py -o ./atlarge/messages/ atlarge/messages/messages.conf
libatlarge.so: $(LIBOBJS)
$(CXX) $(LDFLAGS) -o $# $^ $(LDLIBS)
clean:
#rm -f *.o
#rm -f atlarge/*.o
#rm -f atlarge/messages/*.o
#rm -f atlarge/messages/*.cpp
#rm -f atlarge/messages/*.h
#rm -f atlarge/messages/Makefile.inc
#rm -f atlarge/*.d
#rm -f atlarge/messages/*.d
#rm -f *.d
#rm -f ../common/*.d
#rm -f ../common/*.o
#rm -f *.a
#rm -f *.so
#rm -f tags
install: libatlarge.so
#install -m 0644 $^ $(PREFIX)/lib
#install -m 0755 -d $(PREFIX)/include/atlarge
#install -m 0755 -d $(PREFIX)/include/atlarge/messages
#install -m 0644 -D atlarge/*.h $(PREFIX)/include/atlarge
#install -m 0644 -D atlarge/messages/*.h $(PREFIX)/include/atlarge/messages
#ldconfig
#echo "Installed"
.PHONY: all clean install splint messages
As you can see, I first include the generated Makefile.inc. Then a variable with all library object files is defined, and this variable makes use of the variable declared in the generated Makefile.inc. After that some variables with compiler flags are declared.
To make use of Makefile remaking, I included a target rule for the generated Makefile.inc, so if the dependency of Makefile.inc (the configuration file) is newer than Makefile.inc, it gets regenerated, and Make will restart itself.
So this is the goal:
Check if Makefile.inc needs to be (re)generated.
Include it
Use the variable inside Makefile.inc in the $LIBOBJS variable in the main Makefile.
And this actually works. If I update the messages.conf file, Make detects that, and will run the python script. It will then restart itself, include the new Makefile.inc, and then proceed with compiling.
But here comes the part that doesn't work: if I don't update the messages.conf file, but only .h or .cpp files which are by default in the $LIBOBJS list, Make will not proceed to compile.
For example, if alter client.cpp and no other files, I get the following error:
make: `atlarge/exceptions.o' is up to date.
Well yeah, great you found out that exceptions.o is up to date, but I altered client.cpp, so why don't you start compiling that one? Why does make quit immediatly after seeing that the first target in LIBOBJS is up to date?
Who knows what's causing this, and what could be a solution? Is there maybe a better way to handle code generation with makefiles?
Thanks in advance.
NB: I also use dependency files generated by gcc, and that was working fine before I added the code generation, so I don't think that's a problem.
You need to move the all target to come BEFORE the include. Make always builds the first target it sees in the makefile unless you give a specific target on the command line. Since the include comes before any target, the first target defined in Makefile.inc will be the default target and when you run make that's the one that will be built. That's why it tries to build exceptions.o and then stops. If you run make all explicitly, it will work as you expect.
Where is the dependency for the object files on the source files
or the header files? There is an implicit rule which should
pick up the dependencies if the .cpp file is in the same
directory as the .o, but if they're not, you'll have to
provide your own, or use VPATH (see §4.5.1 in the manual). And
you also need to generate the dependencies for the includes, see
§4.1.4 in the manual.

"Standard library" for building C/C++ projects with Make

Sometimes I need my project to use plain makefile, although it's a dated building technology not recommended for any use, but since make is available almost everywhere it sometimes makes sense.
However I want my Makefile to look something like
Include "../buildexec.mk"
TARG = my_exec
CPPFILES = file1.cpp \
file2.cpp \
and have all the generic (and horrible) dependency tracking code being in buildexec.mk.
Is there such a "Make library"?
In Go, there's a standard Makefile you can include, and your Makefile looks as beautiful like this:
include $(GOROOT)/src/Make.inc
TARG=irc
GOFILES=irc.go irc_struct.go irc_callback.go
include $(GOROOT)/src/Make.pkg
Anything similar for C++?
clarification: I know of tup, cmake and scons/ I know of waf and bjam and so on/ but I want my deps to so small/ so that compilation is no trouble at all. I specifically asked for Make support, not for Make alternative.
I would suggest generating the makefile with automake instead. Cmake also generates makefiles, while also being able to generate project files for several IDEs.
The problem is there is no one make. The very basic syntax is always the same, but anything slightly more complex (which you need for such includes) is not compatible even between GNU make and BSD make (not trying to mention nmake). Automake can however deal with several versions of make and provide automatic dependency rules where available (only some compilers support them).
You probably want to look at CMake or Premake.
With large C/C++ projects you tend to have additional requirements such as platform specific APIs or libraries that need customization.
This will do what you're asking for (at least in GNUMake).
makelib.mk:
$(TARG): $(CPPFILES:.cc=.o)
$(CC) $^ -o $#
%.o : %.cc
$(CC) -MD -o $# $<
#cp $*.d $*.P; \
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $*.d >> $*.P; \
rm -f $*.d
-include $(CPPFILES:.cc=.P)
Makefile:
CPPFILES = foo.cc bar.cc
TARG = someTarget
include makelib.mk # Note lower-case "i"
CPPFILES = baz.cc quartz.cc quince.cc
TARG = anotherTarget
include makelib.mk
...
But a word of advice: don't curse your tools so much. It's energy you could be using to either learn how to use them or switch to ones you like better.
There's a googlecode project doing exactly what I wanted:
http://code.google.com/p/nonrec-make/
There is none of my knowledge. There are really a lot of problems with make, but it is still the most available tool around, and once you get it running properly you should just focus on your development.
Until there, you will have to write makefiles. But instead of looking for a 'make library', a really simple solution (if you're actually initiated in Make language), often overlooked for being too obvious, is to implement your own. Create a make script containing a set of default pattern rules and variables for communication with the project makefile, and just include this script in every project's makefile. It's not hard, maybe a little time-consuming, but it often pays off really well, specially if you have lots of small projects to manage.
I work with such a design. I have a couple of GNU make scripts carefully designed to offer an almost trivial mechanism to create rather complex build systems: automatic dependency generation, handling of different languages, generation of language parsers, different build configurations (debug or release), build log generation, and so on. And the script is not cumbersome: the current version contains just about 250 lines of makefile code, excluding comments.
I will leave you with a sample of an older version of such system, handling C source code only, which contained a few features. It should handle the compilation of binaries and libraries (both static and dynamic). It also should help you track inter-project dependencies through the DEPS variable.
Call this $(ROOT)/project.mk:
# Remove the default suffix rules.
.SUFFIXES:
# Turn on the delete-on-error feature.
.DELETE_ON_ERROR:
# Set up additional command variables.
STRIP ?= strip
# Set up a global search path to locate prerequisites.
VPATH := $(VPATH) $(shell find -type d)
# Locate all source files from the default locations in the project tree.
SRC := $(SRC) $(shell find src -name '*.c')
# Set up the default dependency files.
DEP := $(DEP) $(addprefix dep/,$(addsuffix .d,$(basename $(notdir $(filter %.c,$(SRC))))))
# Set up the default object files.
OBJ := $(OBJ) $(addprefix obj/,$(addsuffix .o,$(basename $(notdir $(filter %.c,$(SRC))))))
# Set up a set of default flags for all commands used.
STRIPFLAGS ?= -p
CPPFLAGS ?= -DNDEBUG
CFLAGS ?= -Wall -Wextra -Werror -pedantic -O3 -march=native -fomit-frame-pointer -ffast-math
LDFLAGS ?= --as-needed -O1
ARFLAGS ?= -scr
# Set up the default include and library search paths.
override INCLUDES := \
$(addprefix $(ROOT)/,$(addsuffix /include,$(DEPS))) \
$(INCLUDES)
override LIBRARIES := \
$(addprefix $(ROOT)/,$(addsuffix /lib,$(DEPS))) \
$(LIBRARIES) lib
# The default rule to build every target in the project.
.PHONY: all
all: deps $(DEP) $(OBJ) $(BIN) $(LIB)
# Phony rule to recursively build the library dependencies.
.PHONY: deps
deps:
#for dep in $(DEPS); do cd $(ROOT)/lib/$$dep && $(MAKE); done
# Secondary expansion is used to properly locate prerequisites.
.SECONDEXPANSION:
# Rule for dependency file generation.
%.d: $$(notdir $$*).c
$(CC) -M $(CPPFLAGS) $(CFLAGS) -iquote include $(addprefix -I ,$(INCLUDES)) $< -MM -MG -MP -MT '$# $(filter %/$(notdir $*).o,$(OBJ))' > $#
# Rule for compiling object files.
%.o: $$(notdir $$*).c
$(CC) -c $(CPPFLAGS) $(CFLAGS) -iquote include $(addprefix -I ,$(INCLUDES)) $< -o $#
# Rule for linking binaries.
%: $$(notdir $$*).c
$(CC) $(CPPFLAGS) $(CFLAGS) $(addprefix -Xlinker ,$(LDFLAGS)) -iquote include $(addprefix -I ,$(INCLUDES)) $(addprefix -L ,$(LIBRARIES)) $(filter-out $<,$^) -o $# $(addprefix -l,$(LDLIBS))
$(STRIP) $(STRIPFLAGS) $#
# Rule for linking shared libraries.
%.so: $$(notdir $$*).c
$(CC) $(CPPFLAGS) $(CFLAGS) $(addprefix -Xlinker ,$(LDFLAGS)) -iquote include $(addprefix -I ,$(INCLUDES)) $(addprefix -L ,$(LIBRARIES)) $(filter-out $<,$^) -o $# -fpic -shared -Wl,-h,$(notdir $#) $(addprefix -l,$(LDLIBS))
$(STRIP) $(STRIPFLAGS) $#
# Rule for generating static libraries.
%.a:
$(AR) $(ARFLAGS) $# $?
# Include all dependency files and remake them if necessary.
ifneq ($(MAKECMDGOALS),clean)
include $(DEP)
endif
# Phony rule to clean the entire build tree.
.PHONY: clean
clean:
#for dep in $(DEPS); do cd $(ROOT)/lib/$$dep && $(MAKE) clean; done
$(RM) $(DEP) $(OBJ) $(BIN) $(LIB) $(CLEAN)
ROOT contains the path for your projects' directory (working copy of repository, for example), typically exported as an environment variable. You will also need a couple of directories (bin, dep, obj and src) in your projects.
An example Makefile using this system could be:
DEPS := mylib
BIN := bin/test
LIB := lib/libtest.a
include $(ROOT)/project.mk
bin/test: $(OBJ)
lib/libtest.a: obj/test1.o obj/test2.o
That is, you just write the minimum necessary about your project, and let the build system do the rest. You can always explicitly specify the value for a given variable (SRC, for example, or CFLAGS), but if you don't, then it gets a reasonable default.
The above was tailored to my needs, but it should be simple to adapt to yours, while keep things as easy as the examples you've mentioned.