How does the gcc decide what files are to be listed as dependencies to an object file being produced, when it writes a .d file?
I'm seeing different .d files when sysroot path is absolute, and relative.
g++ --MMD --sysroot=absolute_path_to_sysroot -c -o file.o -MF file.d file.cpp
g++ --MMD --sysroot=relative_path_to_sysroot -c -o file.o -MF file.d file.cpp
I'm seeing a weird case, where,
If the compiler path and --sysroot are absolute, then 2 additional header files are being listed as dependencies in the .d file.
If the --sysroot is relative, then 2 additional header files are not being listed as dependencies in the .d file anymore.
This made me wonder, what is the criterion, other than a header being #included, based on which, the .d files are actually written by the compiler, and how would sysroot affect this?
As far as I can tell, it is based on the actual directory where the file is found
The actual "make this a dependancy" happens here, where sysp is set a few lines above based on the directory that it is found in.
http://gcc.gnu.org/viewcvs/gcc/trunk/libcpp/files.c?revision=206293&view=markup#l884
sysp gets set here:
http://gcc.gnu.org/viewcvs/gcc/trunk/gcc/incpath.c?revision=206289&view=markup#l446
Now, it gets complicated if the same directory is both in -I and -isystem directories. In that case, it gets determined by whether the file is included with #include <name> or #include "name" - those using <> are selected only if you use -MD or -M, etc, where the -MMD will ignore anything using <> (or in a directory marked with sysp).
Related
I am compiling a self-written Qt5 application, trying to auto-generate the header dependencies with
g++ -MMD -MF../deps/$*.d -MP
The manpage of g++ says:
-MMD
Like -MD except mention only user header files, not system header files.
However, the generated .d files contain stuff like:
/usr/include/qt5/QtWidgets/QDialog /usr/include/qt5/QtWidgets/qdialog.h
/usr/include/qt5/QtWidgets/qtwidgetsglobal.h
/usr/include/qt5/QtGui/qtguiglobal.h /usr/include/qt5/QtCore/qglobal.h
/usr/include/qt5/QtCore/qconfig.h
/usr/include/qt5/QtCore/qtcore-config.h
...
So basically, all the direct and indirect dependencies on Qt header files
are in the output.What am I getting wrong here?
A lot of the examples I see regarding make files are somewhat inconsistent in regards to what files are considered dependencies of main.o and I was wondering what is the safest and most efficient way of going about creating a makefile.
An example from https://www.tutorialspoint.com/makefile/makefile_quick_guide.htm:
hello: main.o factorial.o hello.o
$(CC) main.o factorial.o hello.o -o hello
main.o: main.cpp functions.h
$(CC) -c main.cpp
factorial.o: factorial.cpp functions.h
$(CC) -c factorial.cpp
hello.o: hello.cpp functions.h
$(CC) -c hello.cpp
As you can see, the header file functions.h is a dependency of main.o.
An example from my textbook:
myprog.exe : main.o threeintsfcts.o
g++ main.o threeintsfcts.o -o myprog.exe
main.o : main.cpp threeintsfcts.cpp threeintsfcts.h
g++ -Wall -c main.cpp
threeintsfcts.o : threeintsfcts.cpp threeintsfcts.h
g++ -Wall -c threeintsfcts.cpp
clean :
rm *.o myprog.exe
As you can see, the header file .h and it's .cpp are dependencies of main.o.
I've also seen another example (from https://www.youtube.com/watch?v=_r7i5X0rXJk) where the only dependency of main.o is main.cpp.
Something like:
myprog.exe : main.o threeintsfcts.o
g++ main.o threeintsfcts.o -o myprog.exe
main.o : main.cpp
g++ -Wall -c main.cpp
threeintsfcts.o : threeintsfcts.cpp threeintsfcts.h
g++ -Wall -c threeintsfcts.cpp
clean :
rm *.o myprog.exe
When a main.cpp includes a .h file, should both the .h and its respective .cpp be included as dependencies?
One of the thoughts that came into my head was this: why should any .h file be included as a dependency anyways? Wouldn't a change in any .h file register as a change in the respective .cpp file since the contents of the .h are just going to be copy and pasted into the respective .cpp file through #include?
I am also unsure of whether to have the respective .cpp as a dependency.
(ex. main.o : main.cpp threeintsfcts.cpp threeintsfcts.h).
I think doing so would go against one of the main benefits of makefiles which is the efficiency of modular compilation. (You would have to recompile the main whenever threeintsfcts.cpp changes).
However, it might make sense to do so in case threeintsfcts.cpp changes the name of one of its functions used in main and you forget to change it in main.
Each object file target needs to depend on its source file, obviously, but also all header files that it is including. The make program itself does not parse the source files, so it doesn't know what headers a source file includes. If one of the header files is missing and modified, the source file will not be automatically recompiled by make.
Because tracking the header file dependencies manually is cumbersome and error-prone, there are tools to automate it, see e.g. this question.
Other source files should however not be dependencies, because one source file should not be including another, so there cannot be any dependency that isn't resolved in the later linker step in the main executable target.
Any change in one source file that would affect a change in the compilation step of another source file would have to be through a change in the former source file's header file which is included in the later one. Therefore the header dependencies are sufficient.
Therefore I see no justification for the textbook example you posted. The first example is fine however, as long as the project size is small enough to track the dependencies manually. The third example is wrong, because it wont recompile main.cpp if the header file changes. (Assuming threeintsfcts.h is included in main.cpp, which is the only thing making sense)
The example from your textbook:
main.o : main.cpp threeintsfcts.cpp threeintsfcts.h
g++ -Wall -c main.cpp
is wrong. The purpose of separating source code into two files is so that they can be compiled independently; if one depends on the other, they have been separated incorrectly.
"...Why should any .h file be included as a dependency anyways?
Wouldn't a change in any .h file register as a change in the
respective .cpp file since the contents of the .h are just going to be
copy and pasted into the respective .cpp file through #include?"
If threeintsfcts.h is the only file that has been changed, then main.cpp has not been changed. Make is not smart enough to parse main.cpp and deduce that threeintsfcts.h ought to be a prerequisite of main.o. (There are ways to get Make to do that, but you must master the basics first.)
...In case threeintsfcts.cpp changes the name of one of its functions
used in main and you forget to change it in main.
In that case you will not be able to build the executable; Make can (and will) inform you of the problem, but not fix it, no matter how you arrange the prerequisite lists.
No you shouldn't have .cpp dependency to main, the whole point of Makefile is for separate compilation.
If you were to include .cpp as a dependency to main, then every time the implementation of that cpp changes main and the cpp would get recompiled which is not what we want. Rather, we only want the cpp file to get re-compiled and main stay the same.
I think that the example from your textbook is a mistake.
I could be wrong, Makefiles is an old friend for me, so I want to know what others have to say on the matter.
I'm answering based on what I personally do and this is what also makes sense to me.
I have the following question. After a successful compilation, if I compile it again after I only change some content in one of the .h files, the computer says:
make: Nothing to be done for `all'.
Can I force the compiler to compile again even if I have only modified the .h files?
If you want your output to be updated when header files change, then you have to add it to your dependency statement:
myprogram: myprogram.cpp myprogam.h
c++ -o myprogram myprogram.cpp
Typically we don't do it this way because the code that does things stays in the cpp file. If you are on unix and want to force a rebuild, just touch a cpp file to update its timestamp (forcing a rebuild) with "touch myprogram.cpp", for example, or delete your existing executable.
If you are using make without a Makefile, letting it infer dependencies, it will probably not infer the header dependencies. In this case, either blow away your executable or touch your source file.
Sounds like your Makefile does not have dependencies configured correctly. That is what you should look into fixing.
If you really want to just force a rebuild rather than fix the underlying problem. Then you can do a make clean before your make all or, if the Makefile does not have a "clean" target, delete all the generated object files and libs/executables and then run make all again.
You can force make to rebuild everything using the --always-make command line option.
However, it sounds like you don't have your dependencies setup properly in your Makefile. If your code (.cpp files) actually include headers, then generally your target for compiling them should have a prerequisite on the header files that it includes.
There is a simpler way than the accepted answer. Simply add -MD to your compiler flags in your Makefile, and -include myfile.d at the end of the Makefile (listing all source files with a *.d extension instead). This will, respectively, generate and reference additional *.d dependency files in your build folder (wherever your *.o files go) when you make, so you do not need to explicitly add every single header file to your makefile dependencies.
This is useful for projects with a long list of header files. Furthermore, this way, you know that you can't forget to include a header file in your Makefile dependencies, preventing troubleshooting time lost later when you think your binary updated when you changed a header file, but it actually didn't because you forgot to put it in the Makefile.
For example, use gcc -MD -I. -c myfile.cpp -o obj/myfile.o, and you can keep your Makefile dependencies as just foo: myfile.cpp without myfile.h.
A shortcut way to do this so you only need to list all files once is something like the following:
# Beginning of Makefile etc. etc.
# Only need to list all files once, right here.
SRCS = myfile.cpp myfile2.cpp
OBJS = $(SRCS:%.cpp=%.o)
# put .o and .d files in ./obj/
# (Assumes 'obj' directory exists)
FULLOBJS = $(addprefix obj/,$(OBJS))
# rule to make object (*.o) files
$(FULLOBJS): obj/%.o:%.cpp
gcc -MD -I. -c %< -o $#
# rule to make binary
foo: $(FULLOBJS)
g++ -o $# $(FULLOBJS)
# rule to clean (Note that it also deletes *.d files)
.PHONY: clean
clean:
rm -rf obj/*.o obj/*.d foo
# include dependency files (*.d) if available
-include $(FULLOBJS:%.o=%.d)
Can I force the compiler to compile again even if I have only modified
the .h files?
Yes ... but you probably want to improve your make (tool).
What I do is to force the most recent compile of the file in question, where the command generated by make shows.
Example:
# ... noise
g++ -O3 -ggdb -std=c++14 -Wall -Wextra -Wshadow -Wnon-virtual-dtor -pedantic -Wcast-align -Wcast-qual -Wconversion -Wpointer-arith -Wunused -Woverloaded-virtual -O0 lmbm101_11.cc -o lmbm101_11 -L../../bag -lbag_i686 -lnet_i686 -lposix_i686 -lzlib_i686 -lrt -pthread
# ... more noise.
To force a build, I highlight the "command" make created (starts with "g++", and resides between noise and more noise), and invoke it instead of make.
This is trivial using emacs on Linux. Might not be so easy on other systems.
You might consider copying this command into the file, for future use.
( i.e. I bypass make until I choose to fix my make file. )
I have main.cpp file for which I want to generate dependency file main.d (to be included in Makefile).
I'm calling g++ -MM -MF src/main.d -MP -MT src/main.o src/main.cpp. It works fine unless I have some source generator. I found -MG flag for this, but it doesn't work as I expected it to work.
For example: main.cpp includes module/mod.h (#include "module/mod.h") and mod.h includes generator/gen.h (#include "generator/gen.h"), gen.h is generated. There is following structure:
-Makefile
-src
--main.cpp
--module
---mod.h
---generator
----gen.h
In Makefile I have a rule to generate src/module/generator/gen.h.
Generated by g++ dependencies for existing files are correct:
src/main.o: src/module/mod.h
But for non-existing file gen.h dependency is just generator/gen.h.
If I generate this file before generating dependencies it is correct src/module/generator/gen.h.
Is there any other option for g++ to correctly generate dependencies for non-existing files?
main.d generated when gen.h doesn't exist:
src/main.o: src/module/mod.h generator/gen.h
Correct main.d generated when gen.h exist:
src/main.o: src/module/mod.h src/module/generator/gen.h
As stated in man gcc
The dependency filename is taken directly from the "#include" directive without prepending path
actually there's no way for gcc to know the real path.
I'd recommend you to fix the include statement to keep include path in sync with make path, i.e. white in mod.h
#include <module/generator/gen.h>
put Makefile to src and add proper -I flag to gcc, e.g.
SRCDIR := $(shell pwd)
CXXFLAGS += -I$(SRCDIR)
it is a good practice to always start all non-local includes from some 'source root'
In the project I have:
main.cpp
template.sth
much more
For each .cpp file I am generating .o file.
Thanks to that I could write simple rule for all .o targets (simplified, a little bit pseudcode version for more clarity):
OBJS = #list of all .o files needed
%.o: %.cpp
g++ -MM -MF %.d -MP -MT %.o %.cpp
g++ -c -o %.o %.cpp
Then I am including all existing .d files, so after each generation I am refreshing dependencies.
It worked unless I had template.sth. This file contains some template for generating h files and cpp files.
When one file, i.e. main.cpp includes file generated from template.sth (lets say gen.h):
Instruction generating .d file doesn't work, because gen.h is missing:
fatal error: gen.h: No such file or directory include "gen.h"
Even if these instructions would work there is a problem with my "workflow". Untill now I could generate .d file for next make. It worked, because adding new dependencies require to change one of current dependecnies. So after adding one .o is rebuilding and new .d is generated. Now I need to detect that before making .o I need to generate gen.h from template.sth.
Is there any way to do it automatically? Problem 1. could be solved if there is some way to tell g++ that if some .h file is missing it can just add it to dependencies.
After solving problem 1. executing make multiple times (I think twice is always enough) end up with built project (first make would generate dependencies files, then second make sees that main.cpp depends on gen.h, gen.h is missing and there is instruction how to create gen.h so it will create gen.h before building main.o).
If it can't be done somehow automatically, how can it be solved? Can I write in Makefile instructions which will build all generated files before any other or I need to manually add this generated file as dependencies in all .o instructions?
UPDATE:
After few changes, with -MG flag g++ generates correct files even for gen.h. I can build my project now with two make commands.
First one will create correct .d files and break, because gen.h is missing.
Second one will have .d files ready, so it will generate gen.h before building main.o, so building main.o will be successful.
Is there a way to generate .d file and then use it, before generating .o?
You need to add dependencies for the generated files:
gen.h: template.sth
...command to create gen.h from template.sth
If you have lots of generated files, you might want a way of creating those dependencies automatically, which would depend on how you are generating them.
In addition, you want to generate the .d files independently from the .o files and use the -MG flag while generating them to ignore missing files:
%.d: %.cpp
g++ -MM -MF $# -MP -MG $<
-include $(OBJS:.o=.d)
This way, the first time you run make, it will generate the .d files with the dependencies and then reread them and recompute all the dependencies before trying to build anything else.