I have a project for which I regularly modify headers and when I do so, and forget to make clean then make, I get all sorts of weird behavior. I'm currently using Qt Creator as my IDE, but I've seen this happen on a Qt-independent project. My project is getting fairly large, and having to rebuild every time I make a header change is becoming unproductive. Any thoughts?
For future reference:
If using the QMake system:
DEPENDPATH += . \
HeaderLocation1/ \
HeaderLocation2/ \
HeaderLocation2/HeaderSubLocation1/ \
HeaderLocation2/HeaderSubLocation2/ \
HeaderLocation2/HeaderSubLocation3/ \
HeaderLocation2/HeaderSubLocation4/ \
HeaderLocation2/HeaderSubLocation5/ \
HeaderLocation3/ \
HeaderLocation3/HeaderSubLocation1/ \
HeaderLocation3/HeaderSubLocation2/ \
Re-run qmake. This will generate a new Makefile which will have proper dependencies.
Example:
A file file.h looking like the following:
#include "some.h"
#include "header.h"
#include "files.h"
...
and file.cpp looking like the following:
#include "file.h"
...
and having in your .pro:
HEADERS += file.h some.h header.h files.h
SOURCES += file.cpp
will produce the following in the resulting Makefile:
file.o: ../src/file.cpp ../src/file.h \
../src/some.h \
../src/header.h \
../src/files.h
$(CXX) -c $(CXXFLAGS) $(INCPATH) -o file.o ../src/file.cpp
Dont know anything about your IDE, and not sure if its relevant, and since you are not including your makefile - I'll just state the obvious - do you use any auto-generated dependencies?
For g++ I use the -MD flag, then in the makefile include $(wildcard bin/*.d) (depending on where your object file is created, mine are created in 'bin')
Also make sure to delete the dep file on a clean build
The solution is to have proper header dependency's in your makefile.
1) Use makedepend to generate the dependency files. You would add a target to your makefile which regenerates the dependency file, and you'd want to invoke that before actually doing your compilation.
2) GCC only: Use '-MMD' and '-MP' options on your compile line for .c/.cpp files. This causes GCC to generate a dependency file for input file. Then, you can include these in your makefile. The advantage here is that with those two options, as you add and remove headers, it should behave as you expect.
This is mainly caused by dependencies between files. So if you alter a .h file which is included in another, that file will also need to be recompiled. So either you need to reduce your includes, or do your coding in .h/.cpp form when possible to make changes in .cpp more often than .h.
Here is an excerpt from my own makefile generating and using dependency files automatically created during compilation. You would have to make an additional entry for cpp files
%.o : %.c
$(CC) -M $(CFLAGS) -o $*.P $<
#cp $*.P $*.d; \
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $*.P >> $*.d; \
rm -f $*.P
$(CC) $(CFLAGS) -c $< -o $#
-include *.d
Dont forget to add delete *.d files on clean build
Related
I'm currently working on an SDL2 project and the directory is structured in the following way
./
|__assets
| |__*.png
|__src
| |__physcis
| |_*.cpp *.hpp
| |__textures
| |_obstacles
| |_*.cpp *.hpp
| |_constants
| |_*.cpp *.hpp
|
|__Makefile
Currently, my makefile has a very simple structure
Makefile:
ROOTDIR=src/
TXTURDIR = src/textures/constant/
OBSTACLEDIR = src/textures/obstacles/
PHYSDIR = src/physics/
OBJS = $(ROOTDIR)Main.cpp \
$(ROOTDIR)WindowInit.cpp \
$(ROOTDIR)Timer.cpp \
$(ROOTDIR)GameLoop.cpp \
$(PHYSDIR)Gravity.cpp \
$(TXTURDIR)Texture.cpp \
$(TXTURDIR)TextureContainer.cpp \
$(TXTURDIR)Ball.cpp \
$(TXTURDIR)Bob.cpp \
$(TXTURDIR)Text.cpp \
$(TXTURDIR)ScoreCounter.cpp \
$(TXTURDIR)FPSCounter.cpp
CC = g++
COMPILER_FLAGS = -g -o
LINKER_FLAGS = -lSDL2 -lSDL2_image -lSDL2_ttf
OUT = exe
all: $(OUT)
$(OUT): $(OBJS)
$(CC) $(COMPILER_FLAGS) $# $^ ${LINKER_FLAGS}
clean:
rm exe
Is there any way to speed my making process up by only compiling certain folders when there is a change and then linking compiled sections together afterwards?
Yes, this is the exact thing Makefiles are designed to do. Here's a set of changes to your existing Makefile that will do it:
Change your OBJS variable to refer to .o files instead of .cpp files. This turns your existing compilation rule into a link rule.
OBJS = $(ROOTDIR)Main.o \
$(ROOTDIR)WindowInit.o \
$(ROOTDIR)Timer.o \
# ... etc ...
Make already knows how to create .o files from the .cpp files, thanks to its built-in set of implicit rules. However, you do need to adjust your configuration variable names to what Make's implicit rules expect. Don't put the -o option in the compiler flags, Make will add that itself.
# _instead of_ setting CC, COMPILER_FLAGS, LINKER_FLAGS
CXX = g++
CXXFLAGS = -g
LIBS = -lSDL2 -lSDL2_image -lSDL2_ttf
Adjust the link rule to match the adjusted variable names. (You don't have a CPPFLAGS right now but you may want it in the future.)
$(OUT): $(OBJS)
$(CXX) $(CXXFLAGS) $(CPPFLAGS) -o $# $^ $(LIBS)
At the bottom of the file, add a set of rules without recipes, specifying the individual dependencies of each object file. This is how you arrange for things to get rebuilt when you change your header files, and it also works around a misfeature where sometimes Make will delete object files immediately after they are used. (For a project this size, it's easiest to keep track of which source files use which header files manually. When it gets big enough that you want to have the computer deal with that for you, look into automake.)
$(ROOTDIR)Main.o: $(ROOTDIR)Main.cpp foo.h bar.h
$(ROOTDIR)WindowInit.o: $(ROOTDIR)WindowInit.cpp foo.h bar.h
# ... etc ...
Change the clean target to clean up the object files as well (and while you're at it, use -rm -f instead of bare rm).
clean:
-rm -f $(OUT) $(OBJS)
Add a .PHONY annotation at the very bottom of the file: this is not strictly necessary but will prevent weird things from happening if you ever have a file named 'all' or 'clean' for some reason:
.PHONY: all clean
And that's it, you're done.
I am trying to make a project using a library named BigInt. My file structure is:
/Users/wen/Projects/challenge/fibonacci3/fibonacci3.cpp
/Users/wen/Projects/challenge/fibonacci3/Makefile
/Users/wen/Projects/include/bigint/<.cc files and .hh files>
/Users/wen/Projects/include/bigint/Makefile
The Fibonacci3 Makefile is as of
LD_FLAGS =
CC_FLAGS =
# Include libraries
include /Users/wen/Projects/include/bigint/Makefile
# Build object files
%.o: %.cc %.cpp $(library-cpp)
g++ -c -o $# $(CC_FLAGS)
# Link object files
fibonacci3: fibonacci3.o $(library-objects)
g++ -o $# $(LD_FLAGS)
and the bigint Makefile is as of (shortened)
# Mention default target.
all:
# Implicit rule to compile C++ files. Modify to your taste.
%.o: %.cc
g++ -c -O2 -Wall -Wextra -pedantic $<
# Components of the library.
library-cpp = \
BigUnsigned.cc \
BigInteger.cc \
BigIntegerAlgorithms.cc \
BigUnsignedInABase.cc \
BigIntegerUtils.cc \
library-objects = \
BigUnsigned.o \
BigInteger.o \
BigIntegerAlgorithms.o \
BigUnsignedInABase.o \
BigIntegerUtils.o \
library-headers = \
NumberlikeArray.hh \
BigUnsigned.hh \
BigInteger.hh \
BigIntegerAlgorithms.hh \
BigUnsignedInABase.hh \
BigIntegerLibrary.hh \
however, make reports that it could not find a rule for a header file?
make: *** No rule to make target `NumberlikeArray.hh', needed by `BigUnsigned.o'. Stop.
[Finished in 0.0s with exit code 2]
What is happening here? Headers are supposed to be included, not compiled, so why is make asking for one?
Thanks in advance!
Solution:
Instead of including the makefile, compile the sources in my own makefile. It worked! Thanks again.
The make program expects all files to be in the current directory. Since you include the second makefile into the current makefile, all files in that are relative to the current directory as well. You have to make sure that the files in the included makefile contains the correct paths.
Make gives this error when it cannot find a dependency; it then tries to build that dependency, but knows no rule to make this header file.
You might have mistyped the header file's name.
I have couple of cpp and hpp files in directory ./src. I compile all cpp files in one binary file, say ./bin/run. I want to re-compile only if I need i.e it or one of its header was changed.
I, probably, can create Makefile where file will be recompiled if and only if it was changed, but it's quite uncomfortable because big part of my code is in the headers. (It's not going to be changed, because the product is header itself and cpp files are tests).
I want to store temporary .o files in ./build
I know about g++ -MM function but I'm not sure how to use it.
I'll glad to see solutions that use not necessary make but any other system possible if they are easy enough.
UPD
I'll try to clarify, what's the problem is:
New cpp's maybe created, includes may be added or gone, etc. I don't want to edit my makefile each time.
To solve the problem I mentioned (-include is not a good solution), I use something like this:
build/%.o: %.cpp
#$(CC) -MD -c -Wall -o $# $<
#cp build/$*.d build/$*.P
#sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < build/$*.P >> build/$*.d
#rm build/$*.P
-include build/*.d
No `%.d rule is needed.
EDIT:
#JackKelly has [*cough*, *choke*] shown me a better way to get effectively the same dependency file:
build/%.o: %.cpp
#$(CC) -MD -c -Wall -o $# $<
#$(CC) -MM -MP -Wall -o $*.d $<
-include build/*.d
Ye, you can have multiple rules for the same target, as long as only one of them has commands; the prerequisites accumulate. The idea is to get a file like this:
file.o: file.cpp headerfile.h
headerfile.h:
The second line (headerfile.h:) is a rule for headerfile.h that has no prerequisites or commands. It does nothing, but it's a rule, so if headerfile.h is missing, Make is satisfied.
You mention g++ -MM, which can do what you're trying to do:
include $(ALLOBJ:%.o=%.d)
%.d: %.cxx
#echo making dependencies for $<
#g++ -MM -MP $(CXXFLAGS) $< -o $#
#sed -i 's,$*\.o,& $# ,g' $#
Basically this defines a rule that creates .d files from .cxx files. The .d files are, in turn, required by the include statement, which requires one for each .o file in ALLOBJ.
The last line in the dependency rule is the 'sed magic' that makes the dependency files regenerate themselves. If you think regular expressions are hacks at best, and evil more often than not, you can use the -MT flag.
The outline of a solution is as follows:
Use auxiliary dependency files for each source file (that is, create foo.dep for foo.c, bar.dep for bar.c etc)
Use gcc -MM to create the dependency files
In order to force make to do this automatically, use foo.c as a prerequisite for foo.dep and foo.o; this requires some minor sed magic on the output of gcc -MM
Include all the dependency files in your main makefile; this is a key step that makes this approach possible.
The last step is written as follows:
-include $(dependency_files)
This is very tricky but possible; see the GNU make manual for more information.
You can do this with Make, you just need to specify the headers in your rule's sensitivity list. For example
myfile.o: myfile.cpp
gcc -c myfile.o myfile.cpp ${LDFLAGS} # This is optional. make can infer this line.
Turns into
myfile.o: myfile.cpp myfile.h
gcc -c myfile.o myfile.cpp ${LDFLAGS} # Again, optional.
Now, whenever myfile.h changes, myfile.cpp will be rebuild. More headers can be chained in a similar way.
I'm using a couple boost libraries and using the rule below to generate automatic dependencies. I think boost headers really slow down the compilation because without the dependency includes in the Makefile, it is abour 10 times faster to compile the project. Is there a way to increase the speed with generated dependencies?
%.o: %.cc
$(CXX) $(CFLAGS) $(INCLUDES) -MD -c $< -o $#
#mv $*.d .deps/
#cp .deps/$*.d .deps/$*.tmp
#sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$$;;' \
-e '/^$$/d' -e 's;$$; :;' < .deps/$*.tmp >> .deps/$*.d
#rm .deps/$*.tmp
As you are likely not going to change the boost headers, there is no reason, that you add them as dependencies. By using -MMD to generate the dependency files, system headers should be ignored and thus your dependency files should get smaller, this means make have to scan less of your hard drive to see if files are still up to date etc. Of course you have to include the headers with <> not "".
What approach do C++ programmers on Unix platform use to create and manage Makefiles?
I was using hand made Makefiles for my projects but they don't handle header file changes and other dependencies. I googled around and found a good solution here.
But I ran into a problem here in the sed command -
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $*.d >> $*.P; \
The problem is with the 3rd expression "-e 's/ *\$$//'.
It doesn't work. Its supposed to remove trailing backslashes. I understand that there has to be double dollar there since this is part of a Makefile. Can someone tell me what wrong here?
Here's the complete Makefile -
CC=g++
CFLAGS=-g -Wall
LIBS=-lpthread
OBJS=file1.o file2.o
TARGET=testProg
$(TARGET) : $(OBJS)
$(CC) -o $# $^ $(CFLAGS) $(LIBS)
%.o : %.cpp
$(CC) -MMD -c -o $# $< $(CFLAGS)
#cp $*.d $*.P; \
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $*.d >> $*.P; \
rm -f $*.d
-include $(OBJS:%.o=%.P)
clean :
rm -f $(TARGET) $(OBJS)
all : $(TARGET)
Apart from the solution to this problem, I would also like some tips/pointers to my 1st question.
gcc/g++ can generate dependencies for you with the -M family of options.
The following works by specifying how to generate .depends files given a source file. By doing -include $(DEPS) $(DEPS) is recognized as a target and will be built/rebuilt when the source files change.
CXX = g++
CXXFLAGS = -Wall -O3
LDFLAGS =
TARGET = testcpp
SRCS = main.cc x.cc foo.cc
OBJS = $(SRCS:.cc=.o)
DEPS = $(SRCS:.cc=.depends)
.PHONY: clean all
all: $(TARGET)
$(TARGET): $(OBJS)
$(CXX) $(CXXFLAGS) $(LDFLAGS) $(OBJS) -o $(TARGET)
.cc.o:
$(CXX) $(CXXFLAGS) -c $< -o $#
%.depends: %.cc
$(CXX) -M $(CXXFLAGS) $< > $#
clean:
rm -f $(OBJS) $(DEPS) $(TARGET)
-include $(DEPS)
I use that approach too and can't praise it highly enough. And I write my makefiles by hand and reuse them a lot on new projects.
.The expression "s/ *\\$//" will work outside the context of Make. Within a makefile it doesn't work because Make tries to interpret "$/" before handing the result to the shell. So you must use "s/ *\\$$//" (note the extra $) within the makefile, but this won't work outside the context of Make (so testing it is a slight pain).
EDIT:
I've tried your makefile, and that sed statement seems to remove trailing backslashes just fine. Try something simpler, like this:
backslash:
#echo " \\" > $#
test: backslash
#echo without sed:
#cat backslash
#echo with sed:
#sed -e 's/ *\\$$//' < backslash
EDIT:
All right, now I'm hooked. Could you try these experiments and tell us the results?
Change the last character to 'z' : s/.$/z/
Change a trailing backslash to 'z' : s/\\$/z/
Change a trailing backslash to 'z' : sm\\$mzm
Delete a trailing backslash : s/\\$//
Delete spaces and a trailing backslash: s/ *\\$//
Try all of these inside and outside of Make, with '$' and '$$'.
In a make file anything you list on the dependency line is a dependency header files or other files included.
BSD tutorial on make Note: you can auto generate header dependency info with the -MM switch of GCC.
I must be missing something. Why doesn't generating dependency files work for you?
I prefer to use CMake, even though it's not strictly the solution to your problem.
It's a project description language that'll generate your Makefiles, Visual Studio Project, Eclipse Project, KDevelop, etc for you. All the dependencies are done for you:
CMakeLists.txt
add_executable(my_exe file1.c file2.c)
target_link_libraries(my_exe my_library)
add_subdirectory(lib)
In lib/CMakeLists.txt
add_library(my_library file3.c file4.c)
This creates a my_exe from file1.c file2.c linked against my_library. I find this much simpler. It also has things like package discovery:
find_package(Qt4)
The makedepend utility is installed on many systems and can be quite useful for generating dependency information.
Here is an example Makefile that uses the include directive (plus a little Perl magic) to incorporate the output from makedepend:
# the name of the executable that we'll build
TARGET = foo_prog
# our .cc source files
SRCS = foo.cc main.cc
# the .o versions of our source files
OBJS := $(patsubst %.cc, %.o, $(filter %.cc, $(SRCS)))
# some flags for compiling
CXXFLAGS = -Wall -Werror
# In order to build $(TARGET), we first build each of the $(OBJS).
# Then we use the given command to link those $(OBJS) into our
# $(TARGET) executable. $^ is a shortcut for $(OBJS). $# is a
# shortcut for $(TARGET).
#
# The default compile rule will compile each of the $(OBJS) for us.
$(TARGET): $(OBJS)
$(CXX) $(CXXFLAGS) $^ -o $#
# Use "make clean" to remove all of the support files.
clean:
rm -f $(OBJS) $(TARGET) Makefile.depend *~
# This automatically uses the 'makedepend' utility to add any
# dependencies that our source files have, namely .h files. This way,
# if the .h files change, the code will be re-compiled.
include Makefile.depend
Makefile.depend: $(SRCS)
makedepend -f- -Y $(SRCS) 2> /dev/null | \
perl -p -e "s/(^.*?:)/Makefile.depend \1/" > Makefile.depend
If both foo.cc and main.cc depend on foo.h, then the contents of Makefile.depend would be:
Makefile.depend foo.o: foo.h
Makefile.depend main.o: foo.h
The end result is that the dependency information from makedepend is injected into the Makefile as a series of rules. It's similar to the approach of using a .d file for each .cc file, but keeps the dependency information in one file instead of scattered all over the place.
In Mozilla's build system, we use GCC's -MD switch to generate the dependency files:
http://mxr.mozilla.org/mozilla-central/source/configure.in#7134
and then we use a script called mddepend.pl to check for removed header files, such that
removing a header simply causes a rebuild, not an error:
http://mxr.mozilla.org/mozilla-central/source/config/rules.mk#2066
http://mxr.mozilla.org/mozilla-central/source/build/unix/mddepend.pl
That script generates an .all.pp file containing all the dependencies, with extra foo.o: FORCE dependencies stuck in for missing header files. We then simply -include the .all.pp file in rules.mk right below there.
You can use qmake to generate Makefiles for a project even if that project is not using Qt.
I use BSD make (pmake?) which does lot of work for me (my lang is C, but I think no difference here). This is my common 'local.prog.mk', I never change it:
.PHONY: tags .depend
# .depend depends only on $(SRCS) in bsd.dep.mk, so we can't track changes of
# header's own dependencies properly. so .depend is .PHONY target here.
CSTD ?=c99
WARNS ?=9
.if !empty(PC_LIST)
PC_CF !=pkg-config --cflags $(PC_LIST)
PC_LD !=pkg-config --libs $(PC_LIST)
.endif
CFLAGS +=$(PC_CF) -fgnu89-inline
.if !defined(NO_DEBUG)
CFLAGS +=-O0 -ggdb3
.endif
LDFLAGS +=$(PC_LD)
CTAGS =exctags
NO_MAN=
NO_OBJ=
CLEANFILES+=$(PROG).core
.include <bsd.prog.mk>
$(PROG): $(SUBDIR)
build: clean cleandepend depend all
run: $(PROG)
./$(PROG)
Note 'bsd.prog.mk' inclusion -- this handles all, build, depend, clean targets. Project-specific BSDmakefiles are simple:
.SILENT:
PROG =hello
SRCS =hello.c world.c
PC_LIST =gtk+-2.0 gnet-2.0
.include "../local.prog.mk"
proto:
cproto -siv `pkg-config --cflags $(PC_LIST)` $(SRCS) > prototypes
CLEANFILES+=prototypes
I just make depend every time I insert/remove any #include directives.
Instead of the sed scripts, use gcc's -MT option to modify the target of the generated dependency rules. This blog post has more info.
With a more modern version of GCC, you can add the -MP flag to have GCC generate empty rules for the headers itself.
I top tip that I have found useful when building dependency files is to include the dependency file as a target in the generated rule:
file.d file.o : file.c header.h header2.h ...
Thus make will regenerate the dependencies if the source or any of the headers change. Including phony targets for the headers (GCC -MP) should then allow stable builds when headers are removed - the absense of required header remains a compilation error, not a make dependency error.
Assuming that dependency files are generated into the same directory as the object files, the following should work for GCC on Unix:
-include $(OBJ:.o=.d)
$(OBJDIR)/%d : $(SRCDIR)/%.cpp
mkdir -p $(#D)
echo -n "$# " > $#.tmp
$(CXX) $(CPPFLAGS) -MM -MP -MT $(#:.d=.o) $< >> $#.tmp
mv $#.tmp $#
(from memory)