Hello (I am using Windows, mingw g++ compiler and mingw32-make)
To generalize my question I would like to learn how to write a c++ source file as follows:
Assuming that foo.cpp depends on foo.h where foo.cpp is in src\ and foo.h is in include\
// foo.cpp
#include "foo.h"
Normally I would just write it like this
//foo.cpp
#include "..\include\foo.h"
but I have found that as my project grows, and I begin to need more organization, that this method isn't dynamic enough. Reason being I have to change every include for every file if I want to move foo.h to a new directory (say include\bar\foo.h). Is there a way for make to achieve this. If so can it be done for header file dependencies as well.
As a side note I am new to makefiles. I am not even sure that it knows these includes are there since they are within the code (in fact from what I understand it doesn't). That would lead me to an unfortunate secondary question, which is can make see these includes? If not is it possible to change it so that it can? Feel free to answer how you would approach this problem because I have a feeling I am going about this the wrong way by putting the includes in the file rather than linking them in the makefile.
The compiler is always looking into some default paths to look for .h-files. You can add your path.
For example gcc takes multiple -I arguments which contain a path. In your foo.cpp you do:
#include "foo.h"
and when compiling you say:
g++ -I../include foo.cpp -c [other options]
.
Regarding the second part of your question: The makefile and the call to make does not normally know anything about the files to be compiled and about your project. However there are several default variables and directives in make which lead to that impression: It could be, that in your environment you only need to change the CFLAGS or CPPFLAGS variable to add the -I-argument and it will work.
Patrick B has answered very well on how to make the compiler know where to include from, but not the following bit:
As a side note I am new to makefiles. I am not even sure that it knows
these includes are there since they are within the code (in fact from
what I understand it doesn't). That would lead me to an unfortunate
secondary question, which is can make see these includes? If not is it
possible to change it so that it can?
No, make doesn't understand what your source files contain, or how they depend on other files [make also doesn't really care if you are programming in C, C++, Fortran, Pascal, ADA, Lisp, Cobol or Haskell - as long as there is a "If you have a file like this, and want a file like that by doing something" relationship between files, make will sort it for you.
There are several ways to do this. You can manually add:
foo.cpp: foo.h
Or you can use a dependency file for your include-file, and let make built it automatically, by adding this, for example:
SOURCES = foo.cpp # Add any further source files here.
INCLUDES = -I../includes # Add other include directories if needed.
CFLAGS += ${INCLUDES}
TARGET = foo.exe # in Windows. Just foo in linux/MacOS.
all: ${TARGET} deps.mk
${TARGET}: ${SOURCES}
gcc -O $# $^
desp.mk: ${SOURCES}
gcc -MM ${INCLUDES} $^ > $#
include deps.mk
Note that makefiles are RELYING on indentation being tabs. This post uses spaces, so you will need to "tabify" the recepies. Also note that in a "proper" makefile, you'd make foo.o from foo.cpp, etc, and link all the different .o files together. That way, the compile is a fair bit quicker for large projects. I've simplified it for readability.
Maybe I should expand a little bit:
gcc -MM gives a list (to standard out) of the files that are being "compiled" and all of it's dependencies. It doesn't actually compile the code (and as long as the code is at least SOMEWHAT) close to being compileable, it will happily process your files.
For more details on gcc -MM and related, have a look at the GCC invocation documentation.
The $# and $&^ are what make calls "Automatic variables" - they expand to the "target" (easy to remember, as it looks sort of like a target for shooting arrows at or similar) and "all dependencies" (no visual clue here, I'm afraid - and every now and again, I have to remind myself) respectively. Check out here for more details.
Related
I'm trying to compile some C++ sources with GNU make using implicit rules, here's the content of my Makefile:
%.o: %.cpp
g++ -o $# $<
and the output of make
make: *** No targets. Stop.
Running make -d isn't very helping (I don't post the output which is quite verbose other than apparently useless). The folder is not empty.
You told Make how to build .o files, but you didn't tell it:
to build any .o files;
which .o files to build;
what to do with those .o files afterwards.
The rule doesn't mean that, when invoked without arguments, all .cpp files in the current directory or something will be compiled.
This one rule is just not enough.
You told make how to create arbitrary .o files from corresponding .cpp files, but you didn't tell it which files to create.
Analogy: I tell you, that, say, to make USB sticks, you'll need plastic, memory chips and a USB-A plug (now you know how to create any USB stick). But I don't say which USB sticks to create. This is exactly, what's happening here.
To solve your issue, you can say to make to create (for example) "asdf.o" by invoking make asdf.o, and it'll gladly do so, by using g++ -o asdf.o asdf.cpp.
You can also add an (de facto standard) all target, like so:
all: asdf.o fdsa.o [you get it]
And since the first target is the default, calling make is equivalent to calling make all, which in turn requires "asdf.o" and "fdsa.o", which will be created as described above.
We are refactoring our code base, and trying to limit the direct dependencies between different components. Our source tree has several top level directories: src/a, src/b and src/c.
We want to enforce a set of restirctions:
Files in a cannot depend of files in b or c
Files in b can depend on files a but not c
Files in c can directly depend on files in b but not a
Enforcing the first one is simple. I have an implicit rule like this:
build/a/%.o : src/a/%.cpp
$(CXX) -I src/a $(OTHER_FLAGS) -o $# $<
If a file under a tries to include a header file from b or c, the build fails as the header is not found.
The second rule has a similar rule, which specifies src/a and src/b as include directories. The problem arises with building c. The following is allowed.
src/c/C.cpp
#include "b.h"
void C() { ... }
src/b/b.h
#include "a.h"
class B { ... };
src/a/a.h
class A { ... };
Here, a file from c includes a file from b (allowed), which in turn includes a file from a (also allowed). We want to prevent code like this:
src/c/C_bad.cpp
// Direct inclusion of a
#include "a.h"
src/c/c_bad.h
// Direct inclusion of a
#include "a.h"
For the allowed case to compile, the compile command for building files in src/c must include a -Isrc/a, but that allows the second cases to also compile.
I suspect that the answer to my problem is writing a script which looks at the dependencies generated from the compiler, finds potentially illegal dependencies and then looks at the source files to determine if this is a direct dependency. Is there a reasonable way to do this combining the compiler and/or makefile constructs?
If it matters, we are using GNU Make 3.81 and g++ 4.5.3, but would like to be portable if possible.
Update
We are looking for something where it takes effort to violate the rules, not one where it takes effort to follow the rules. (Past experience has shown that the latter is unlikely to work.) While there are some good ideas in the other answer, I'm accepting the one that says to write a script, since that is the one that takes the most effort to work around.
Thanks to everyone for your answers.
Considering the fact that you're applying this on an existing code base, I would opt for the "validation script" approach.
So instead of modifying the build process and severing dependencies one at a time as the build fails, you get presented with a list of files that are non-complaint. You can then refactor your codebase having the "big picture" in mind and any changes you make will be built using the same Makefiles as before thus simplifying testing and debugging.
Once refactored, the analysis script can continue to be used as a compliance checker to validate future updates.
A possible starting point for such an analysis would be to use makedepend or cpp -MM. For example, using the cpp/h files you've listed in the question:
[me#home]$ find .
.
./b
./b/b.h
./a
./a/a.h
./c
./c/C_bad.cpp
./c/C.cpp
./c/c_bad.h
[me#home]$ cpp -MM -Ia -Ib -Ic */*.cpp
C_bad.o: c/C_bad.cpp a/a.h
C.o: c/C.cpp b/b.h a/a.h
[me#home]$ # This also works for header files
[me#home]$ cpp -Ia -Ib -Ic -MM c/c_bad.h
c_bad.o: c/c_bad.h a/a.h
It should be reasonably straight-forward to parse those output to determine the dependencies of each cpp file and flag up those that are non-compliant.
The drawback to this approach is that it cannot differentiate between direct and indirect dependencies, so if that matters you may need to include an extra step to inspect the source and pick out direct dependencies.
You can make the -I options target-specific:
build/b/%.o: CPPFLAGS += -Isrc/a
build/c/%.o: CPPFLAGS += -Isrc/b
This is specific to gnu-make, though, so it's not portable.
Yes. But it takes some manual effort and discipline.
When building C you can depend on headers in src/b/*.h.
Inside project B any header files in the main directory should be self-contained and not have dependencies on other projects. You also need a subdirectory inside B src/b/detail/*.h. In here header files are allowed to include src/a/*.h and src/b/*.h but this is a private implementation detail and only available to source files for the b project.
The easiest way is to change your include path to -Isrc for everything. Include statements then have the complete relative path
#include <a/a.h>
for example. This makes it much easier to check the code automatically (perhaps in a commit hook rather than the makefile).
Alternatively, you could do something nasty with macros in the A and B headers:
// src/a/a.h
#ifndef SRC_A_H
#define SRC_A_H
#ifndef ALLOW_A
#error "you're not allowed to include A headers here"
#endif
//...
and
// src/b/b.h
#ifndef SRC_B_H
#define SRC_B_H
#ifdef ALLOW_A_INDIRECT
#define ALLOW_A
#endif
#include <a/a.h>
//...
#ifdef ALLOW_A_INDIRECT
#undef ALLOW_A
#endif
#endif // include guard
Now these make rules will allow A and B to build ok:
build/a/%.o: CPPFLAGS += -DALLOW_A
build/b/%.o: CPPFLAGS += -DALLOW_A
and this will allow C access only via B (and the macros in B's headers)
build/c/%.o: CPPFLAGS += -DALLOW_A_INDIRECT
Note this requires some discipline especially in B's headers, but I suppose if it sits alongside existing include guards, it ... ok, it's actually still pretty nasty.
I have a main code (.cpp) and some other files (.cpp and their corresponding .h file) which contain the function that are used in my main program. I'd like to know how to compile my files and run the main code.
This depends on your compiler. For example, with GCC you might write:
g++ foo.cpp bar.cpp baz.cpp -o foo # compile program
./foo # run program
Basile's right on, you need build machinery to have software you can easily work on. You really want to be able to just run make or something similar and get your project re-built quickly in a repeatable fashion. You also want to make sure that your entire project is kept up to date, rebuilding everything that depends upon something that has changed. While you could just keep around a shell script containing g++ lines as in ruakh's answer, this will needlessly recompile more than you need. (Which isn't a big deal with small projects but becomes important when projects grow larger than trivial.)
I've only taken fifteen minutes to skim the OMake documentation, but it looks very promising. make is standard on systems and provides a huge array of pre-defined build rules, including rules for C++, so it isn't very difficult to write new Makefiles for projects.
It'll look something like this:
# the compiler and its flags
CXX = g++
CXXFLAGS = -Wall
PROGRAM = foo
OBJECTS = foo_a.o foo_b.o foo_c.o foo_d.o
.PHONY: all
.DEFAULT: all
all: $(PROGRAM)
$(PROGRAM): $(OBJECTS)
The objects foo_a.o will be built from a corresponding source file named foo_a.c, foo_a.cc, foo_a.C, foo_a.cpp, foo_a.p, foo_a.f, foo_a.F, etc., for C, C++, Fortran, Pascal, Lex, Yacc, and so on.
The O'Reilly make book is sadly quite dated at this point; the .SUFFIX rules have been supplanted by pattern rules, but there is no coverage of pattern rules in the O'Reilly book. That said, it is still a good starting point if the full GNU Make documentation isn't a good fit for you.
But the real answer is to use some build machinery, like e.g. have a Makefile and use GNU Make.
There are some better builder than GNU Make, like Omake
Builders are very useful, because in the example by ruakh you may don't want to recompile foo.cpp when only bar.cpp changed so you need to take dependencies into account.
(and there are Makefile generators, like automake, cmake ...)
Sometimes when I change code in my Qt project (Qt Creator 2.1.0 with mingw32), the changes don't get reflected in the produced machine code after building it. This happens mostly when I change things like default values in constructors or the order of parameters in methods/constructors. Usually, a full rebuild fixes that (but takes a few minutes).
I'm helping myself by deleting the generated executables or librarys before building, which seems to help most of the time. Does that mean that theres something going wrong when linking the object files?
I'm coming from java/.net and I'm used to a different behaviour. I'd be happy if anyone could explain me what I'm doing wrong and/or point me to some related articles.
Thank you!
Usually, after a change in a header, all source files including that header should be rebuilt.
However, qmake is a bit peculiar in this regard, you need to set DEPENDPATH for include folders other than the current directory. E.g., if you have
INCLUDEPATH += somepath_in_my_project
also add
DEPENDPATH += some_path_in_my_project
Only with DEPENDPATH, files built by the .pro files are rebuilt if some header in some_path_in_my_project changes (if they include that header)!
I suggest to add for each INCLUDEPATH line an identical DEPENDPATH line, unless you include some system directory you don't expect to change.
Edit:
A similar problem exists when linking statically with qmake: If the static lib foo.a changes, binaries linking against it are not relinked. That's a bug in QMake, not generating the correct dependencies.
A workaround I found in a former project:
static:unix:TARGETDEPS += path_to_my/somestaticlib.a
static:win32:TARGETDEPS += path_to_my/somestaticlib.lib
Edit edit:
Since some time (Qt 5?), above code should use POST_TARGETDEPS instead of TARGETDEPS.
The most common case for that are broken dependencies. In the particular case of default arguments to functions, they are resolved at the place of call, so if you just recompile the function, the code will be exactly the same. You need to recompile the caller. If the dependencies in the project are not correct and the build system does not detect that it needs to recompile the caller, and only recompiles the callee then you will see that effect.
Analyze the dependencies and fix them.
Example:
// what you write // what the compiler generates
void foo( int i = 0 ) {} void foo( int i ) {} // default removed
int main() { int main() {
foo(); foo( 0 ); // compiler injected
} }
If you're listing all relevant header files in a project file this shouldn't happen. But actually it happens all the time because QMAKE is buggy (it has known problems with dependencies generation unfixed for years). So better clean it and recompile or use Cmake. And also QMAKE knows nothing (and detect almost nothing) about dependencies between source files and header files and that may lead to problems like that.
If your Makefile (or your moral equivalent to a Makefile) is missing dependency information, then you can get out-of-sync builds. For every file with #include "header.h", you need to make sure the Makefile puts header.h as a dependency on that file.
Here is a small piece of a Makefile I've got:
parser_yacc.c parser_yacc.h: parser_yacc.y parser.h
$(YACC) $(YFLAGS) -o parser_yacc.c parser_yacc.y
parser_lex.c: parser_lex.l parser_yacc.h parser.h
$(LEX) ${LEXFLAGS} -o$# $<
parser_lex.o: parser_lex.c parser.h parser_yacc.h
$(CC) $(EXTRA_CFLAGS) -c -o $# $<
parser_misc.o: parser_misc.c parser.h parser_yacc.h af_names.h cap_names.h
$(CC) $(EXTRA_CFLAGS) -c -o $# $<
Each object file clearly depends upon the corresponding source and header files. If we forgot cap_names.h, then parser_misc.o wouldn't be rebuilt when cap_names.h is modified. Bugs can ensue.
I'm afraid the fix is long and tedious: inspect every file, make a list of its dependencies, and add missing dependencies to the Makefile. On a Linux platform, you could use the strace(1) tool to discover which files the compiler needs to open when compiling each source file, and improve the dependencies with that list. I do not know if any similar tool exists on Windows, but it would be worth spending a few minutes looking for one before diving in.
I'm going to use a small example for reference. Consider a project with:
inner_definitions.o : inner_definitions.cpp inner_definitions.h
gcc $^ -o $#
inner_class_1.o : inner_class_1.cpp inner_class_1.h inner_definitions.h
gcc $^ -o $#
inner_class_2.o : inner_class_2.cpp inner_class_2.h inner_definitions.h
gcc $^ -o $#
outer_class.o : outer_class.cpp outer_class.h inner_class_1.h inner_class_2.h
gcc $^ -o $#
executable.o : executable.cpp executable.h outer_class.h
gcc $^ -o $#
executable : __?1__
__?2__
But filling in the blanks __?1__ for the linker dependencies and __?2__ for the linker command isn't easy.
In this small example, one could argue that its easy to see that __?1__ = inner_definitions.o inner_class_1.o inner_class_2.o outer_class.o executable.o . However, this is clearly not a scalable solution as it forces each developer to understand all the dependencies of the code they are working with so they can figure out the dependencies by hand rather than by using the make utility.
Another solution would be to have a different variable for each object file that listed all its downstream dependencies: i.e __?1__ = executable.o $(executable_dependencies). This is not a desired solution because it forces the makefile to be compiled in the specific way so the variables are only used when they are fully defined. Also, for really large applications these variables might exceed the maximum variable length.
Yet another solution is to use archive .a files for linking. In this case, we could construct an inner_class_1.a that included both inner_defintions.o and inner_class_1.o, so it could be linked with any object file that needed inner_class_1.o without forcing the developer to reconstruct the dependencies. This approach seems promising, but involves having many duplicate files. Also, it doesn't appear that the gcc linker can handle nested archive files.
Is there another approach? What is the best approach? Can the gcc linker handle nested archive files?
The job you're trying to automate (picking the right object files to satisfy all references) is usually left to the linker, using static libraries (".a" files) to group the candidate object files, just as you suggest.
An important detail you may be missing: If you pass the linker an archive, it will only link in those files from the archive that are actually needed. So you can create archives at a fairly coarse level of granularity without necessarily bloating all your executables -- the linker will pick just what it needs -- although can easily end up with needlessly slow builds if you take this approach too far.
The GNU linker will not pull objects out of nested libraries. If you want to make one big library by merging many small ones, you can do that with the "addlib" command in an ar script. That will give you a single archive containing all of the object files without any nested library structure.
If the duplication of having .o files and .a files containing the same object code lying around bothers you, the fine manual describes a way to have make update the archives "directly".
Your makefile must have a list of objects to link together, like so:
OBJ_FILES = inner_definitions.o inner_class_1.o inner_class_2.o \
outer_class.o executable.o
executable : $(OBJ_FILES)
gcc $^ -o $#
Someone must write this; gcc can't do it for you, Make can't do it for you. Not every developer on the project needs to know how to construct that list, only the one who writes that part of the makefile. All the others will use that makefile, and a developer who adds a new dependency (e.g. inner_class_3) can add it to the list.
And if your makefile is lost in a fire and the only developer who knows all the dependencies is hit by a bus, it really isn't hard to reconstruct the list: when you try to make executable, the linker complains that foo::bar() is undefined, you grep around and discover that foo::bar() is defined in inner_class_2.cpp, you add inner_class_2.o to the list. Repeat until the linker stops complaining.
P.S. Once that's in order, you can simplify the rest of the makefile quite a lot:
%.o: %.cpp %.h
gcc -c $< -o $#
inner_class_1.o inner_class_2.o : inner_definitions.h
outer_class.o : inner_class_1.h inner_class_2.h
executable.o : outer_class.h
EDIT:
The method I suggested does not require listing every object file that can be made, just the ones that are actually needed to build `executable`; I inferred the list from your question.
Passing extra object files to the linker makes no difference to the final executable, but it does lead to unnecessary rebuilding. For example, suppose you add `alien.o` to `OBJ_FILES`. Then if you modify `alien.cpp` and run `make executable`, it will rebuild `alien.o` and `executable` even though there's no real need to do so. Correction (thanks to slowdog): unnecessary object files go into the final executable as dead code-- but I'm still right about unnecessary rebuilding.
Organizing object files into archives and shared libraries is often convenient, but doesn't really change anything here.
I know of no robust way to automatically construct the object list -- that is, a way that could deal with problem cases such as when the same function is defined in two different source files. This could become a real problem if unit tests are involved. But you could do it within your makefile if you follow a simple naming convention.
The trick for doing it within your makefile is a pretty advanced one. I honestly think you'd be better off doing this the simple way until you're more comfortable with the tools.
EDIT:
All right, here's an outline of the advanced technique.
First, consider all of the #included header files. It would be nice to have Make handle the dependencies instead of putting them in by hand, as in the makefile above. And this is a straightforward task: if X.cpp #includes Y.h (either directly or through some chain of #included header files), then X.o will depend on Y.h. This has already been worked out as "Advanced Auto-Dependency Generation". But if you follow a strict naming convention, you can take it a step further: if everything declared but not defines in X.h is defined in X.cpp, then by following the same tree of #include statements we should be able to construct a list of the needed object files, which will then be the dependencies of executable.
This really is a lot to absorb at once, so I won't try to work through an example. I suggest you look over the document and see how it can generate the Y.h dependencies, then try applying it to the example makefile, then think about what the "step further" should do.
Later you can apply it to the test harness, where the object files are, e.g., outer_class.o, stub_inner_class_1.o, stub_inner_class_2.o and test_outer_class.o.