Working with Makefile, source generators and generating dependencies with gcc - c++

In the project I have:
main.cpp
template.sth
much more
For each .cpp file I am generating .o file.
Thanks to that I could write simple rule for all .o targets (simplified, a little bit pseudcode version for more clarity):
OBJS = #list of all .o files needed
%.o: %.cpp
g++ -MM -MF %.d -MP -MT %.o %.cpp
g++ -c -o %.o %.cpp
Then I am including all existing .d files, so after each generation I am refreshing dependencies.
It worked unless I had template.sth. This file contains some template for generating h files and cpp files.
When one file, i.e. main.cpp includes file generated from template.sth (lets say gen.h):
Instruction generating .d file doesn't work, because gen.h is missing:
fatal error: gen.h: No such file or directory include "gen.h"
Even if these instructions would work there is a problem with my "workflow". Untill now I could generate .d file for next make. It worked, because adding new dependencies require to change one of current dependecnies. So after adding one .o is rebuilding and new .d is generated. Now I need to detect that before making .o I need to generate gen.h from template.sth.
Is there any way to do it automatically? Problem 1. could be solved if there is some way to tell g++ that if some .h file is missing it can just add it to dependencies.
After solving problem 1. executing make multiple times (I think twice is always enough) end up with built project (first make would generate dependencies files, then second make sees that main.cpp depends on gen.h, gen.h is missing and there is instruction how to create gen.h so it will create gen.h before building main.o).
If it can't be done somehow automatically, how can it be solved? Can I write in Makefile instructions which will build all generated files before any other or I need to manually add this generated file as dependencies in all .o instructions?
UPDATE:
After few changes, with -MG flag g++ generates correct files even for gen.h. I can build my project now with two make commands.
First one will create correct .d files and break, because gen.h is missing.
Second one will have .d files ready, so it will generate gen.h before building main.o, so building main.o will be successful.
Is there a way to generate .d file and then use it, before generating .o?

You need to add dependencies for the generated files:
gen.h: template.sth
...command to create gen.h from template.sth
If you have lots of generated files, you might want a way of creating those dependencies automatically, which would depend on how you are generating them.
In addition, you want to generate the .d files independently from the .o files and use the -MG flag while generating them to ignore missing files:
%.d: %.cpp
g++ -MM -MF $# -MP -MG $<
-include $(OBJS:.o=.d)
This way, the first time you run make, it will generate the .d files with the dependencies and then reread them and recompute all the dependencies before trying to build anything else.

Related

Changed .h file in C++ does not need to be compiled again?

I have the following question. After a successful compilation, if I compile it again after I only change some content in one of the .h files, the computer says:
make: Nothing to be done for `all'.
Can I force the compiler to compile again even if I have only modified the .h files?
If you want your output to be updated when header files change, then you have to add it to your dependency statement:
myprogram: myprogram.cpp myprogam.h
c++ -o myprogram myprogram.cpp
Typically we don't do it this way because the code that does things stays in the cpp file. If you are on unix and want to force a rebuild, just touch a cpp file to update its timestamp (forcing a rebuild) with "touch myprogram.cpp", for example, or delete your existing executable.
If you are using make without a Makefile, letting it infer dependencies, it will probably not infer the header dependencies. In this case, either blow away your executable or touch your source file.
Sounds like your Makefile does not have dependencies configured correctly. That is what you should look into fixing.
If you really want to just force a rebuild rather than fix the underlying problem. Then you can do a make clean before your make all or, if the Makefile does not have a "clean" target, delete all the generated object files and libs/executables and then run make all again.
You can force make to rebuild everything using the --always-make command line option.
However, it sounds like you don't have your dependencies setup properly in your Makefile. If your code (.cpp files) actually include headers, then generally your target for compiling them should have a prerequisite on the header files that it includes.
There is a simpler way than the accepted answer. Simply add -MD to your compiler flags in your Makefile, and -include myfile.d at the end of the Makefile (listing all source files with a *.d extension instead). This will, respectively, generate and reference additional *.d dependency files in your build folder (wherever your *.o files go) when you make, so you do not need to explicitly add every single header file to your makefile dependencies.
This is useful for projects with a long list of header files. Furthermore, this way, you know that you can't forget to include a header file in your Makefile dependencies, preventing troubleshooting time lost later when you think your binary updated when you changed a header file, but it actually didn't because you forgot to put it in the Makefile.
For example, use gcc -MD -I. -c myfile.cpp -o obj/myfile.o, and you can keep your Makefile dependencies as just foo: myfile.cpp without myfile.h.
A shortcut way to do this so you only need to list all files once is something like the following:
# Beginning of Makefile etc. etc.
# Only need to list all files once, right here.
SRCS = myfile.cpp myfile2.cpp
OBJS = $(SRCS:%.cpp=%.o)
# put .o and .d files in ./obj/
# (Assumes 'obj' directory exists)
FULLOBJS = $(addprefix obj/,$(OBJS))
# rule to make object (*.o) files
$(FULLOBJS): obj/%.o:%.cpp
gcc -MD -I. -c %< -o $#
# rule to make binary
foo: $(FULLOBJS)
g++ -o $# $(FULLOBJS)
# rule to clean (Note that it also deletes *.d files)
.PHONY: clean
clean:
rm -rf obj/*.o obj/*.d foo
# include dependency files (*.d) if available
-include $(FULLOBJS:%.o=%.d)
Can I force the compiler to compile again even if I have only modified
the .h files?
Yes ... but you probably want to improve your make (tool).
What I do is to force the most recent compile of the file in question, where the command generated by make shows.
Example:
# ... noise
g++ -O3 -ggdb -std=c++14 -Wall -Wextra -Wshadow -Wnon-virtual-dtor -pedantic -Wcast-align -Wcast-qual -Wconversion -Wpointer-arith -Wunused -Woverloaded-virtual -O0 lmbm101_11.cc -o lmbm101_11 -L../../bag -lbag_i686 -lnet_i686 -lposix_i686 -lzlib_i686 -lrt -pthread
# ... more noise.
To force a build, I highlight the "command" make created (starts with "g++", and resides between noise and more noise), and invoke it instead of make.
This is trivial using emacs on Linux. Might not be so easy on other systems.
You might consider copying this command into the file, for future use.
( i.e. I bypass make until I choose to fix my make file. )

Adding dependency on a non-compiled file in a Makefile

I have the following code generation scheme:
The original file is *.c.macro
It is processed by a special program that generates the *.c file. The program uses external XML files mentioned in the original .c.macro file. Somthing along this line:
macroprocess foo.c.macro -o foo.c
Then the *.c file is compiled to *.o normally
What I want to do is to make the .c file dependant on the XML files. So that if the XML changes, macroprocessing is automatically redone when 'make' command is invoked.
I can modify the code of the macroprocessor so that it would generate a list of all XMLs and write it somewhere, but I have no idea how to incorporate it into a makefile. Apparently I would need to play with 'include' command in the makefile, but usually it's employed to add dependencies on actual code like .h files.
If you can get your macroprocess to write a dependency file (e.g. *.xmldep) for each *.c.macro file you can do:
-include $(C_MACRO:.c.macro=.xmldep)
where $(C_MACRO) is a variable that contains your *.c.macro files.
The - in front will ensure that the first time, the make will not fail when it cannot locate the dependency files.
You can add foo.xml as a dependency of foo.c and use a filter command to get only the %.c.macro file as arguments for macroprocess.
Something like this:
# Basic targets
all: foo
foo: foo.o
gcc -o $# $^
foo.o: foo.c
gcc -o $# -c $<
# Special target with filter
foo.c: foo.c.macro
macroprocess -o $# $(filter %.c.macro,#^)
# Adding foo.xml as a dependency of foo.c
foo.c: foo.xml
.PHONY: all foo

Example makefile for building simple c project recompiling when headers change

Does anyone have a complete makefile that can do the following:
Rebuilds the project if a HEADER file changes
The cpp files are listed in the makefile
The header files are NOT listed in the makefile
The header files are allowed to have different names than the cpp files
Some of the cpp files do not have header files
I have seen instructions for figuring out how to make the make tool figure out header dependencies, but none of these instructions provide anything remotely resembling a working example. A simple example would be as follows: main.cpp C1.cpp C1.h C2.cpp C2.h
CXX = g++
OBJECTS := main.o C1.o C2.o
all: $(OBJECTS)
%.o : %.cpp
$(CXX) $(CPPFLAGS) -Wall -MMD -c $< -o $#
-include *.d
EDIT: As TobySpeight points out, this won't work if you build an object file, rename or delete one of the prerequisite source or header files, then try to rebuild the object file; the .d file will still require the missing file, and the build will fail. I neglected to include lines to deal with that case:
%.h: ;
%.cpp: ;
(This is effective, but crude. The more precise approach is to put some sed commands in the %.o rule, so as to add specific null rules to the .d file, one for each prerequisite, but the sed commands are ugly, and the approach above is good enough for almost all cases.)
You can also use CMake for this. Everything you need to write is:
add_executable (exec main.cpp C1.cpp C2.cpp)

Managing Dependency Complexity in Make Files

I am working on my first open source C++ project: https://github.com/jehugaleahsa/spider-cpp.
I am managing my own Makefile and I have "best practices" question regarding how to manage dependencies. Right now, I make each .o file dependent on each of the included header files in the .cpp file. So:
code.o: code.cpp code.hpp dep1.hpp de2.hpp
g++ -c code.cpp
First of all, I am pretty sure Make supports a shorthand for creating object files. If someone would show an example of this, I'd appreciate it.
Next, is there a way to avoid listing every included header as a dependency? I want to make sure if I change a dependency, that the changes are still compatible. Listing the included headers is tedious and easy to mess up.
OP:
First of all, I am pretty sure Make supports a shorthand for creating object files. If someone would show an example of this, I'd appreciate it.
From here:
OBJS := foo.o bar.o
#Your program should have the objects as dependencies, and link them
proggie: $(OBJS)
gcc $(OBJS) -o proggie
# compile
%.o: %.c
gcc -c $(CFLAGS) $*.c -o $*.o
OP:
Next, is there a way to avoid listing every included header as a dependency
Lower down on the same page, see these lines:
# pull in dependency info for *existing* .o files
-include $(OBJS:.o=.d)
# compile and generate dependency info
%.o: %.c
gcc -c $(CFLAGS) $*.c -o $*.o
gcc -MM $(CFLAGS) $*.c > $*.d
Basically what this does is use gcc's -MM option to obtain a list of header files, and now we can depend on them. Thus we output a file with a list of such header files to a .d file, and then next time, we add the list of files as a dependency, which is what the -include command does. The "-" avoids error if the dependency .d files don't exist yet.
Note, you should modify the above to account for .cpp files
Yes, make supports shorthand for creating object files. This is called rules. And yes, there is a way to avoid listing every included header as a dependency. g++/gcc has -MM option which will generate the full list of dependencies.
Unfortunately, there is no simple explanation on how to do it which I could put here.. You have to read docs and play with make utility. I found this book very helpful: "Managing Projects with GNU Make". There is a doc on GNU site but I found it a bit harder to read. www.gnu.org/software/make/manual/make.html

how to write makefile to take care of changes in the header file

Actually i have a library 'cryptopp' and what i want is that when i make any change to a file and issue the make command it should take care of the changes made in any file in the source directory. well, the GNUMakefile of cryptoopp takes care of the changes 'if' made in the '.cpp' files but not for the changes made in a '.h' file.
So what changes can i make in the 'GNUMakefile' of cryptopp so that it looks at all the modified header files and recompiles all the files dependent on the 'modified' header file.
If you are building with g++ you can let g++ generate dependancy makefiles.
You can include these in your main makefile.
Use the -M and -M* arguments to use this feature. (see http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Preprocessor-Options.html#Preprocessor-Options)
You have to add all the dependencies to your Makefile:
mycode.o: mycode.cpp mycode.h somelib.h resources.h
$(CXX) -c -o $# $< $(CXXFLAGS) $(INCLUDES)
If you already have a generic pattern matching command line, you don't have to say the command again, you can just list the dependencies:
%o: %.cpp
$(CXX) -c -o $# $< $(CXXFLAGS) $(INCLUDES)
mycode.o: mycode.cpp mycode.h somelib.h resources.h
yourcode.o: yourcode.cpp yourcode.h mycode.h somethingelse.h
# ...
In general, this is a terrible and unscalable mess. You'll almost definitely want a higher-level build system to generate the Makefile for you. Even for very small projects keeping the header dependencies up to date in the Makefile is such a pain that it is simply not worth it.
There are several popular portable build environments. I personally like cmake a lot, which includes discovery if you changed the build settings (say from Debug to Release) and will always build all the necessary files (for example, if you change the cmake master file and type "make" it'll automatically run cmake again for you first).
For a Unix-only solution you could try makedepend, or the infamous autotools, though that's a whole other headache...
You might try 'makedepend' if it's installed on your system. The easiest way is to add a target to your makefile. Something like:
depend:
makedepend *.cc
You might have to replace the '*.cc' with a list of your source files. Then you can regenerate all the dependencies with 'make depend' command. You might want to redirect error messages to /dev/null since it always seems to generate a lot of noise.