I recently learned how to use Makefiles, and I found out that GCC/G++ generates dependencies for you:
$ g++ -MM file.cpp
file.o: file.cpp file.h
I then thought that the obvious thing to do would be to use this to generate dependencies directly in the file without creating a dependency file:
CXX = g++
SRCS = $(wildcard src/*.cpp)
OBJS = $(SRCS:.cpp=.o)
OCT = $(CXX -MM $(SRCS))
OBJDIR = obj
CPPFLAGS = -Wall -I/usr/local/include -L/usr/local/lib -lGLEW -lglfw -lGL
.PHONY: all
all: $(OBJS)
$(CXX) $(CPPFLAGS) $(OBJS) -o output
$(OCT)
.PHONY: clean
clean:
rm -f obj/*
For some reason, I have never seen anyone else do this; they always generate a dependency file. Is there something wrong with this system? In my case yes -- the objects don't go to OBJDIR, they go to they location of the source file. I'm sure this could be fixed though. If anyone knows how I could fix this and why dependency files are usually generated, please let me know.
Well, the first reason people don't do this is that it's not possible to do: if you try to make your suggestion work in real life you'll see that. Your example, for instance, doesn't do anything at all. This:
OCT = $(CXX -MM $(SRCS))
(I assume you mean $($(CXX) -MM $(SRCS)) but it doesn't matter either way) is putting a reference to the make variable named, literally, CXX -MM $(SRCS) into the variable OCT: you probably are thinking that it's using the shell command invocation syntax $(...) but this is a makefile, not a shell script. So when you write:
$(OCT)
that tries to look up that make variable, which obviously does not exist, and so this expands to the empty string and nothing at all happens. If you actually try to test your makefile by touching a header etc. you'll see nothing gets rebuilt.
How can you do this? You can't do it. You can change your variable assignment like this:
OCT = $(shell $(CXX) -MM $(SRCS))
and that will actually run the compiler, which is moving you in the right direction, but the results of the shell function will change all newlines to whitespace, so this:
$(OCT)
will expand to the entire output of the compiler command on one line, and since it contains multiple colons you'll get a syntax error.
All you can do is redirect the output of the compiler to a file, and using make's include capability to include that file. But now you're basically back to the scenario suggested in the GNU make manual, except your version is less efficient because, as pointed out in the comments above, you're regenerating all the headers for all the source files every time you run make, rather than only regenerating header information for the files that have actually changed.
There are better/more efficient ways to generate headers, such as the one used by most GNU packages.
Related
This question already has answers here:
How to make Makefile recompile when a header file is changed?
(3 answers)
Closed 3 months ago.
I thought my makefile is pretty good but then I tried to update only a header file and recompile my library, but it doesn't change anything.
How can I make it work and recompile also the code in the h file.
My makefile:
SHELL = /bin/sh
SOURCE_FILES_DIRS = -I./../frmwrk/ -I./../Utils/
CXX = g++
CXXFLAGS = $(SOURCE_FILES_DIRS) -std=c++17 -rdynamic -fPIC -g -Wall
LDFLAGS = -shared
LIBS_DIR = ../../libs/
LIB_NAME = libIni.so
TARGET = $(LIBS_DIR)$(LIB_NAME)
SOURCES = $(shell echo *.cpp)
HEADERS = $(shell echo *.h)
OBJECTS = $(SOURCES:.cpp=.o)
LINK_LIBS = -lFrmwrk -lUtils
PREFIX = $(DESTDIR)/usr/local
BINDIR = $(PREFIX)/bin
all: $(TARGET)
$(TARGET): $(OBJECTS) $(HEADERS)
$(CXX) $(CXXFLAGS) -L$(LIBS_DIR) $(LDFLAGS) -o $(TARGET) $(OBJECTS) $(HEADERS) $(LINK_LIBS)
clean:
rm -f *.o $(TARGET)/*.so
rm -rf $(TARGET)
$(TARGET): $(OBJECTS) $(HEADERS)
[relink command]
This forces only a relink when the header file is changed. make has no idea, whatsoever, which C++ source file includes which header file. make knows only about dependencies explicitly declared in the header file. The only dependency stated here is that $(TARGET), the executable, has a dependency on the header file.
In order to force, for example, main.o to be recompiled when declarations.h are included, because main.cpp #includes declarations.h, you have to be explicit:
main.o: declarations.h
This directs make to rebuild main.o, from main.cpp, whenever declarations.h changes, which is what you want.
Manually tracking which object modules need to be rebuilt due to changes to which header files does not scale. make, of course, has no knowledge about anything C++ related, so you'll have to manually keep track of all the dependencies, manually, but that again doesn't scale.
The solution here is to migrate to some higher level build tools, like GNU autoconf and automake which will write the Makefile for you, complete with a bunch of rules that use compiler flags to dump the dependencies, and update the rules automatically, for you. You can always use the same compiler flags (there are flags that, and build all the scaffolding yourself, without autoconf and automake. But why bother, when autoconf automake will do it for you?
I know that this might be totally late but a quick solution to this would simply be to run a make clean such that you rm (remove command) remove the executables. By doing this I found a quick solution to this sort of problem.
I am new to C++ and am trying to get the hang of build systems like make/CMake. Coming from Go, it seems that there is a constant risk that if you forget to do a little thing, your binaries will become stale. In particular, I can't find a best practice for remembering to keep dependencies/prerequisites updated in make/CMake. I'm hoping I am missing something obvious.
For example, suppose I have a basic makefile that just compiles main.cpp:
CFLAGS = -stdlib=libc++ -std=c++17
main: main.o
clang++ $(CFLAGS) main.o -o main
main.o: main.cpp
clang++ $(CFLAGS) -c main.cpp -o main.o
main.cpp:
#include <iostream>
int main() {
std::cout << "Hello, world\n";
}
So far so good; make works as expected. But suppose I have some other header-only library called cow.cpp:
#include <iostream>
namespace cow {
void moo() {
std::cout << "Moo!\n";
}
}
And I decide to call moo() from within main.cpp via `include "cow.cpp":
#include <iostream>
#include "cow.cpp"
int main() {
std::cout << "Hello, world\n";
cow::moo();
}
However, I forget to update the dependencies for main.o in makefile. This mistake is not revealed during the obvious testing period of running make and rerunning the binary ./main, because the whole cow.cpp library is directly included in main.cpp. So everything seems fine, and Moo! is printed out as expected.
But when I change cow.cpp to print Bark! instead of Moo!, then running make doesn't do anything and now my ./main binary is out of date, and Moo! is still printed from ./main.
I'm very curious to hear how experienced C++ devs avoid this problem with much more complicated codebases. Perhaps if you force yourself to split every file into a header and an implementation file, you'll at least be able to quickly correct all such errors? This doesn't seem bulletproof either; since header files sometimes contain some inline implementations.
My example uses make instead of CMake, but it looks like CMake has the same dependency listing problem in target_link_libraries (though transitivity helps a bit).
As a related question: it seems like the obvious solution is for the build system to just look at the source files and infer dependencies (it can just go one level in and rely on CMake to handle transitivity). Is there a reason this doesn't work? Is there a build system that actually does this, or should I write my own?
Thanks!
First of all you will need to reference the dependencies file in your Makefile.
This can be done with the function
SOURCES := $(wildcard *.cpp)
DEPENDS := $(patsubst %.cpp,%.d,$(SOURCES))
wich will take the name of all *.cpp files and substitute and append the extension *.d to name your dependency.
Then in your code
-include $(DEPENDS)
- tells the Makefile to not complain if the files do not exist. If they exist they will be included and recompile your sources properly according to the dependencies.
Finally the dependencies can be created automatically with the options: -MMD -MP for the rules to create the objects file. Here you can find a complete explanation. What generates the dependencies is MMD; MP is to avoid some errors. If you want to recompile when system libraries are updated use MD instead of MMD.
In your case you can try:
main.o: main.cpp
clang++ $(CFLAGS) -MMD -MP -c main.cpp -o main.o
If you have more files it is better to have a single rule to create object files. Something like:
%.o: %.cpp Makefile
clang++ $(CFLAGS) -MMD -MP -c $< -o $#
You can take a look also at this 2 great answers:
one
two
In your case a more suitable Makefile should look like the following (there might be some errors but let me know):
CXX = clang++
CXXFLAGS = -stdlib=libc++ -std=c++17
WARNING := -Wall -Wextra
PROJDIR := .
SOURCEDIR := $(PROJDIR)/
SOURCES := $(wildcard $(SOURCEDIR)/*.cpp)
OBJDIR := $(PROJDIR)/
OBJECTS := $(patsubst $(SOURCEDIR)/%.cpp,$(OBJDIR)/%.o,$(SOURCES))
DEPENDS := $(patsubst $(SOURCEDIR)/%.cpp,$(OBJDIR)/%.d,$(SOURCES))
# .PHONY means these rules get executed even if
# files of those names exist.
.PHONY: all clean
all: main
clean:
$(RM) $(OBJECTS) $(DEPENDS) main
# Linking the executable from the object files
main: $(OBJECTS)
$(CXX) $(WARNING) $(CXXFLAGS) $^ -o $#
#include your dependencies
-include $(DEPENDS)
#create OBJDIR if not existin (you should not need this)
$(OBJDIR):
mkdir -p $(OBJDIR)
$(OBJDIR)/%.o: $(SOURCEDIR)/%.cpp Makefile | $(OBJDIR)
$(CXX) $(WARNING) $(CXXFLAGS) -MMD -MP -c $< -o $#
EDIT to answer comments
As another question, is there any problem with rewriting the DEPENDS definition as just DEPENDS := $(wildcard $(OBJDIR)/*.d)?
Nice question, it took me a while to see your point
From here
$(wildcard pattern…) This string, used anywhere in a makefile, is
replaced by a space-separated list of names of existing files that
match one of the given file name patterns. If no existing file name
matches a pattern, then that pattern is omitted from the output of the
wildcard function.
So wildcard return a list of the file names matching the pattern. patsubst acts on strings, it does not care about what are those strings: it is used as a way to create the file names of the dependencies, not the files themselves. In the Makefile example that I posted DEPENDS is actually use in two cases: when cleaning with make clean and with include so in this case they both work because you are not using DEPENDS in any rule. There are some differences (I tried to run and you should too to confirm). With DEPENDS := $(patsubst $(SOURCEDIR)/%.cpp,$(OBJDIR)/%.d,$(SOURCES)) if you run make clean dependencies *.d that do not have a correspondent *.cpp file will not be removed while they will with your change. On the contrary you might include dependencies not relevant to your *.cpp file.
I asked this questions: let's see the answers.
If the .d files get deleted from carelessness but the .o files remain, then we are in trouble. In the original example, if main.d is deleted and then cow.cpp is subsequently changed, make won't realize it needs to recompile main.o and thus it will never recreate the dependency file. Is there a way to cheaply create the .d files without recompiling the object files? If so then we could probably recreate all the /.d files on every make command?
Nice question again.
Yes, you are right. Actually it was an error of mine. This happens because of the rule
main: main.o
$(CXX) $(WARNING) $(CFLAGS) main.o -o main
actually should have been:
main: $(OBJECTS)
$(CXX) $(WARNING) $(CXXFLAGS) $^ -o $#
so that it got relinked (the executable is updated) whenever one of the objects change and they will change whenever one their cpp file change.
One problem remains: if you delete your dependencies but not the objects, and change only one or more header files (but not the sources) then your program is not updated.
I corrected also the previous part of the answer.
EDIT 2
To create the dependencies you can also add a new rule to your Makefile:
here is an example.
My directory structure looks like this:
root
|____SG
| |
| |____Makefile
| |____simple_client_main.cpp
|
|___EEE
|___my_utils.h
SG is essentially my base of operations for building "simple_client", and I'm running make from here. In simple_client_main.cpp I have the following #includes:
#include <iostream>
#include <string>
#include "my_utils.h"
So I need my makefile to know where my_utils.h is. With this in mind, I want to add the root/EEE directory as an include directory. (From where I am, that would be ../EEE.)
Following the advice suggested here, my makefile looks like this:
DIR1 = ../EEE
CXXFLAGS = $(FLAG)
OBJS = simple_client_main.o
SRCS = simple_client_main.cpp
all: simple_client
simple_client: $(OBJS)
g++ -o simple_client -I$(DIR1) $(OBJS) -lz
# [...]
depend:
makedepend -- $(CFLAGS) -- $(SRCS)
But it doesn't work:
simple_client_main.cpp:6:25: fatal error: my_utils.h: No such file or directory
compilation terminated.
Note that if I manually set the #include directive in the cpp as follows:
#include "../EEE/my_utils.h"
...everything works as expected.
What am I likely to be doing wrong here?
You need to add -I$(DIR1) to either CFLAGS or CXXFLAGS (or perhaps both), so that when the object file is compiled, the option is present in the compiler command line.
You want make to execute something similar to:
g++ -c -I../EEE simple_client_main.cpp
It should do that if you add -I../EEE to $(CXXFLAGS) or $(CFLAGS). You need to know the rules used by the make program you're using — they can vary.
When linking object files, it is too late for the -I option to be of relevance (but you should still include $(CFLAGS) or $(CXXFLAGS) in the linker command line as other options, notably -g, are of relevance when linking as well as when compiling to object code).
Here is some simple modifications to the outline makefile shown in the question.
DIR1 = ../EEE
IFLAGS = -I$(DIR1)
CXXFLAGS = $(FLAG) $(IFLAGS)
CFLAGS = $(IFLAGS)
LDFLAGS =
LDLIBS = -lz
CXX = g++
OBJS = simple_client_main.o
SRCS = simple_client_main.cpp
all: simple_client
simple_client: $(OBJS)
$(CXX) -o $# $(CXXFLAGS) $(OBJS) $(LDFLAGS) $(LDLIBS)
A makefile like this stands a modest chance of working correctly. It is not clear what you might put in the FLAG macro, so I've left it. (I use UFLAGS and UXXFLAGS for 'user-defined C (or C++) flags'; they can be set on the command line and are never set by the makefile and are included in the CFLAGS or CXXFLAGS — you may be after something similar.)
Note how the linking line is almost all macros. This is normal and desirable; macros can be changed when running make without editing the makefile, but constant text cannot be changed without editing the makefile. The -c and -o options to the C and C++ compilers are about all that should ever appear as plain text.
If there are still problems, look at the built-in rule for compiling C++ source code to an object file, and tweak definitions accordingly. (You can use make -p to print the rules — you may need that to find out what is going on, but I hope not for your sake because they tend to be complex. Using make -f /dev/null -p shows the built-in rules only; that can be useful, too.)
Note that the make depend rule may need some surgery. It uses $(CFLAGS). If $(CXXFLAGS) contains extra options that are needed by the makedepend command, then you may need that instead, or even as well. If you have only C++ source, you probably only need the $(CXXFLAGS) macro in the command line.
Is the error coming from the compile stage, or the makedepend stage?
Because what I see above is that makedepend uses $(CFLAGS), and you haven't put -I$(DIR1) into CFLAGS.
I seem to be having an issue getting my makefile to build my C++ file correctly. My makefile code is below; the file I am trying to compile is named "avl.cc" (which is working and compiles properly).
CC=g++
CFLAGS=-g -O2
PROGS=avl
all: $(PROGS)
$#:
$(CC) $(CFLAGS) $# $#.cc
.PHONY: clean
clean:
rm $(PROGS)
However, when I enter the command make or make all, I get
c++ avl.cc -o avl
And the debugging symbols I want from the -g flag don't come up. A similar makefile (only changing the PROGS variable) worked for a similar project, so I am not sure what I'm doing wrong. Does anyone have any tips? Thanks!
From Makefile documentation about automatic variables:
It’s very important that you recognize the limited scope in which
automatic variable values are available: they only have values within
the recipe. In particular, you cannot use them anywhere within the
target list of a rule; they have no value there and will expand to the
empty string.
This means you cannot use $# as a rule, which means the default c++ compilation rule of Makefile is used, and since you did not use the correct variable names for c++ compilation, they are also ignored.
You can replace CC by CXX and CFLAGS by CXXFLAGS to work with c++.
You don't have a target for 'avl', so make uses a default rule.
Try changing the makefile to this:
CC=g++
CFLAGS=-g -O2
PROGS=avl
all: $(PROGS)
$(PROGS):
$(CC) $(CFLAGS) -o $# $#.cc
.PHONY: clean
clean:
rm $(PROGS)
I had the exact same question but a much different source of the problem. There were typos or misnamed files in my makefile. Make found no rules for such files but tried to compile targets with the c++ compiler. This made the process seem like it was ignoring my rules and imposing its own, switching compilers since I needed g++. Finally I tried using the -r option, and then the resulting different error messages allowed me to figure out what was really wrong. Below is the entry from the make man page for option -r.
-r, --no-builtin-rules
Eliminate use of the built-in implicit rules. Also clear out the default
list of suffixes for suffix rules.
I've followed the instructions online to set up gcc (actually g++) to generate .d files for dependencies, and now my makefile looks something like this:
CPPFLAGS := ... -MMD -MP
...
OBJECTS := $(shell find *.cpp *.s | sed -e 's/\.cpp/\.o/' -e 's/\.s/\.o/')
all: setupdir $(OBJECTS) link image
And then at the end of the file:
-include $(pathsubst %.d,obj/%.d,$(OBJECTS:.o=.d))
The .d files are being correctly generated, and all show up in the obj directory. The problem is, now, even with the assembly files that don't have .d files generated for them the entire source tree is being rebuilt every time I run make. The project doesn't take long to compile, but still, how would one go about fixing it so that make runs correctly?
It's important to note that -M is not magic bullet for dependencies, in many cases you'll need to specify certain headers so things are re-built appropriately if they change. In fact, the most common problem with letting gcc handle dependencies is things not being re-built that should be (broadly depending on the version of GCC being used by whoever runs make).
That being said, I don't see anything blatantly wrong with what you're doing.
What I recommend doing is eliminating the shell tricks, since it's quite easy to specify exactly what you want. E.g.
foo_SOURCES = \
src/foo.cpp \
src/bar.cpp
foo_OBJECTS = $(foo_SOURCES:.cpp=.o)
foo_DEPS = $(foo_OBJECTS:.o=.d)
%.o : %.cpp
$(CC) $(CFLAGS) ... $< -o $#
$(CC) -M $< > $(#.o=.d)
# lots more rules and targets here
# and finally, at the bottom:
-include $(foo_DEPS)
This is off the top of my head in meta form, so check it to be sure. I believe your problem is actually the dependencies not being included, but I don't see anything obviously wrong with the way you are using pathsubst.
As people have noted in comments, running make with debug on may be helpful, but a good first step would be to specify your objects and dependencies without talking to the shell.
If you get that working, try just storing the results of your shell calls and echoing them to the screen. That's a lot less noisy than make's debug output.