Makefile Dependencies, What Should Be a Dependency? - c++

I have a conceptual question regarding makefile dependencies and this is because I see inconsistency online about this.
Let's say I have the following files:
main.cpp uses-> my_math.cpp and my_strings.cpp
my_math.cpp uses-> my_math.h
my_strings.cpp uses-> my_strings.h
If I have a makefile, with the general outlay of:
program: $(all_objs)
g++ $(all_objs) -o program
main.o: ...
.......
my_math.o: ...
.......
my_strings.o: ...
.......
I don't know what should go into each dependency. Like, math.o #includes my_math.h and my_strings.h, does that mean that main.cpp needs to recompile if I change my_math.h? But why? It uses it like a library, right? It doesn't need to recompile main.cpp or does it?
As in, should the result for main.o be:
1) main.o: main.cpp
gcc -c main.cpp
2) main.o: main.cpp my_strings.cpp my_strings.h my_math.cpp my_math.h
gcc -c main.cpp
3) main.o: main.cpp my_strings.cpp my_strings.h my_math.cpp my_math.h
gcc -c main.cpp my_strings.cpp my_math.cpp
I'm a bit lost on dependencies and how linking works.
Any help would be appreciated! Thank you!

The dependencies are everything whose change requires recompiling the source code. That includes not only your #include-d headers, but also the indirectly included system headers, and even (in principle) the compiler and build chain itself (when you upgrade your C++ compiler, you should recompile all your software). If some of your C++ code is generated from some source (e.g. by tools like GNU bison or Qt moc, or by your own script), both the sources and the generating tools are dependencies. Read also about package managers.
Practically speaking, the GCC compiler is able to output most make dependencies, notably with -M and related processor options. Read also about auto dependencies generation. See also this.
(in practice, you generally don't code in your Makefile some explicit dependency on the compiler itself; but you should not forget to make clean when the compiler has been upgraded)
Unless your main.cpp is including my_strings.cpp (which is not conventional and is very bad taste), your make rule won't have a dependency from my_strings.cpp to main.o. But probably your main.cpp is #include-ing (directly or indirectly) my_strings.h so main.o should depend not only on main.cpp but also on my_strings.h
As a rule of thumb, your object file my_strings.o depends on the source file my_strings.cpp and all the header files which are directly or indirectly #include-d in it. Your main program executable depends on all its object files and the libraries you are linking into it. Order of program arguments to g++ matters a lot.
It uses it like a library, right?
From what you are showing, you don't have any own libraries (but you probably use the standard C++ library, and perhaps some other system libraries). On Linux these are lib*.a files (static libraries) or lib*.so files (shared libraries). A library is an organized agglomeration of object code -and sometimes other resources.
I'm a bit lost on dependencies and how linking works.
Understand the difference between source code files, object files (they contain relocation information) and executables (on Linux, object files and executable files and shared libraries are using the ELF format). Read also about the role of compilers, linkers (the g++ program can run both) & build automation (for which you are using make).
Read Program Library HowTo and much more about translation units and linkers (& name mangling), notably Levine's book on Linkers & loaders.
See also this & that & this (examples about Makefile for C++ programs).
BTW, you should use g++ (not gcc) when compiling C++ code. There are significant differences (even if gcc is sometimes able to compile C++ or Fortran code, you'll mostly use gcc to compile C code). And (assuming you use specifically GNU make) your Makefile should mention $(CXX) (not g++). You need to understand the builtin rules of make (run once make -p to get them) and you'll better take advantage of them (e.g. use $(COMPILE.cc) or $(COMPILE.cpp) etc...). You certainly should pass -Wall -Wextra (to get all warnings, and even more), and -g (to get debugging information) to g++. Practically speaking, you should set your CXXFLAGS variable in your Makefile.
Take time to carefully read GNU make documentation and Invoking GCC.
Look into the Makefile-s of existing free software projects. For various reasons, some projects are generating their Makefile-s with tools like autoconf or cmake. But most simple projects don't need that generality, and you should be able to write your own Makefile for your C++ projects. Of course, take inspiration from existing code.

If you have
main.cpp uses-> my_math.cpp and my_strings.cpp
my_math.cpp uses-> my_math.h
my_strings.cpp uses-> my_strings.h
The purpose of Make is to maintain dependency between modules in two different ways, by building .o files and by linking .o files.
you can picture it as a dependency tree where main.o is the root
main.o
/ \
my_math.o my_strings.o
for each .o there is also a dependency tree with regard to source files e.g.
main.o my_math.o my_strings.o
/ \ / \ / \
main.cpp main.h my_math.cpp my_math.h my_strings.cpp my_strings.h
So when make builds, it sets up a dependency tree with main.o at its root and then tries to build all .o files needed for main. When all .o files have been built they are linked.
By following the dependency tree Make ensures that main will be linked/built when one of the dependent modules is changed.
However if you have used something like say a constant from one of the included headers #define MAXSTRING 32 you are no longer merely dependent on the .o file, you are then dependent on the header content so you need to make sure that main.o is built if the header is changed since linking is then not enough so you add the .h in the dependency
main.o
/ | \
my_math.o my_strings.o my_strings.h
Of course there are ways to make the header more robust to avoid that dependency but is another question.

Your cpp files do not depend on other cpp files in terms of compilation. Simple cpp file should only depend on h files.
In your question you say main.cpp depends on my_math.cpp and my_strings.cpp, but I think it is not true. I can guess you have #includes there and these are your dependecies.
Generally speaking dependencies for cpp files are all #included h files.
Usually there are no dependencies between cpp files. You just generate o files by compiling them. Then your final binary depends on all o files.

Related

Building a C++ template library with make

I am having difficulties with re-writting my C++ library into template form; main problem concerns re-designing the Makefile.
In the previous state, when it was non-template I had:
a header file lib.h in which (protected by include guards) we have the declarations of classes and overloaded operators. This file does not include any other libraries at all.
an implementation file lib.cpp in which, on top of the file, I include many headers from the standard library (cmath, iostream, etc) as well as I include the header file for this custom library: #include "lib.h"
a Makefile with commands for build/install the library:
CC = g++
CFLAGS = -O2
SRC = lib.cpp
HDR = $(SRC:.cpp=.h)
OBJ = $(SRC:.cpp=.o)
LIB = $(OBJ:.o=.a)
.PHONY: all install clean uninstall
# =========================================================
# Build
# =========================================================
all: $(LIB)
# Create an object file
$(OBJ):
$(CC) $(CFLAGS) -c -o $# $(SRC)
# Create a static library file
$(LIB): $(OBJ)
ar rcs $# $<
# =========================================================
# Install
# =========================================================
install: ~/lib/lib/$(LIB) ~/lib/include/$(HDR)
# Create top-level directory for the libraries
~/lib:
mkdir -p $#;
# Create top-level directory for the static library files
~/lib/lib:
mkdir -p $#;
# Create top-level directory for the headers
~/lib/include:
mkdir -p $#;
# Copy the library file into the right directory
~/lib/lib/$(LIB): $(LIB) ~/lib/lib
cp $< $#
# Copy the header file into the right directory
~/lib/include/$(HDR): $(HDR) ~/lib/include
cp $< $#
following the installation my CI (GH Actions) would compile a small test program, which includes my library header (#include <lib.h>) with the following command:
g++ -O0 -Wall --std=c++14 test.cpp -I ~/lib/include/ ~/lib/lib/lib.a -o test
This setup worked fine.
Problems emerge now, when I want to re-write my class to be template.
Following the information in this post I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc). With that change I needed to adjust the compilation process and do not provide my implementation file in the command line (as it is already included in the header and I have to avoid re-definition errors). That works fine. The core of the problem is now in the Makefile, in the command to build an object file. As I try to compile the implementation file of my library, it includes the header and the header includes the implementation again... I red about the problem in this port and they suggest to remove the header inclusion from the implementation file. So I did that, I commented the #include "lib.h" and tried to run:
g++ -O2 -c -o lib.o lib.cpp
And I end up with a lot of error: use of undeclared identifier errors...
How can I build the library properly with make? My constraint is that the library stays in two separate files: header and implementation. What I want in the end is to be able to #include <lib.h> in my further programs.
Is it even possible to create an archived object file? What about shared object library (.so)
I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc).
I can't say I think much of the advice you followed there. Code that is intended to be used as or in a header should be named appropriately (.h, .hpp, or whatever convention your project follows), and code that is suitable for direct compilation pretty much never belongs in a header. Perhaps your conversion involves changing everything of the latter kind into to former kind, so maybe you want to rename lib.cpp to lib_impl.h, or something similar, and skip trying to compile it at all. Maybe.
Note well, however, that if you name and structure the implementation code as a header, then it needs its own include guards. Note also that it must not, then, contain any external, non-template functions. If it did, then no two separate translation units contributing to the same program could both include the header, as that would result in duplicate function definitions, include guards notwithstanding. And if there are no external, non-template functions (and no external object definitions) then it is pointless to try to compile to an object file, as there would be no accessible entry point to any function within.
How can I build the library properly with make?
This isn't really about make in particular. That just provides automation. The issue is with the structure of the library itself and your expectations.
My constraint is
that the library stays in two separate files: header and
implementation.
This constraint makes sense only if the implementation contains any instantiated templates for external objects or functions, or any external non-template functions or objects. These are the things that would contribute to a buildable object file that you could directly or indirectly link to applications. If converting to a template library means that there are no longer any such entities in your library then the constraint is arbitrary, as the converted result is a header-only library. Nevertheless, you can split up your headers any way you like, as long as each resulting one is structured as a header, with only contents appropriate for a header, and with its own inclusion guards.
On the other hand, if your converted implementation code does contain any of those things then they must not be #included into any header, as discussed above.
What I want in the end is to be able to #include <lib.h> in my further programs.
If the converted implementation code is suitable for use as or in a header, then you are already there, but it would be much better style to rename your lib.cpp as a header. If you want to be able to include that header directly into code other than the main library header then it needs its own inclusion guards -- that will take care of the duplicate declarations. Do note, however, that those errors arise from the fact that you have a circular dependency, and that's a strong sign that you ought to refactor. Such a refactoring would involve moving enough code from lib.cpp to lib.h that the circular dependency can be removed (one of the files would no longer #include the other).
Whatever implementation code is not suitable for use in a header obviously must not be included into the header. If any such code remains in the converted library then perhaps you keep that in lib.cpp, and move the rest into lib.h
Is it even possible to create an archived object file? What about shared object library (.so)
Templates cannot be compiled. They are templates for code that can be compiled: their instantiations. If your conversion leaves nothing else then no, you cannot usefully create an object file or a shared library. And you don't need to do, for this is a header-only library.

How to use a library with headers and .so files?

I'm new to C and wanted to use a library (MLT Multimedia Framework)
I've built it and it produced the following directories: include lib share
Inside lib there are .so .a .la files
Inside include there are .h files
Now, I'm instructed to do this:
#include <framework/mlt.h> which is inside include/mlt/framework/
Questions:
Why I do I need to place the header file that contains only function prototypes? Where are the real functions then? are they linked someway to the ones included in lib directory?
Where to place my own files and how to compile it?
How to learn more about the topics:
Dynamic/Static libraries
Building / making / installing
How to use any C library
If you don't have the function prototypes, how would the compiler know what functions exist in the library? Short answer is: It doesn't. Longer answer: The compiler doesn't care about library files, static (files ending in .a) or shared (files ending in .so), all it cares about is the current translation unit. It's up to the linker to handle resolving undefined references.
When you use libraries, you include the header files that contain the needed declarations (structures, classes, types, function prototypes) into the source file. The source file plus all included header files forms the translation unit that the compiler uses to generate code. If there are undefined references (for example a call to a function in the library) the compiler adds special information about that to the generated object file. The linker then looks through all object files, and if it finds an unresolved reference it tries to find it in the other object files and the provided libraries. If all definitions are resolved the linker generates the final executable, otherwise it will report the unresolved definitions as errors.
To answer your other questions:
Where to place my own files and how to compile it?
This is two questions, the answer to the first one (about placement of your files) is that it doesn't really matter. For small project with only a few source and header files, it's common to place all files in a common project directory.
The second question, about compiling, there are different ways to do it too. If there are only one or two source files you could use the compiler frontend (e.g. gcc) to compile and link and generate your executable all in one go:
$ gcc -Wall -g source1.c source2.c -o your_program_name
The above command takes two source files, compiles and links them into the program your_program_name.
If you need to use a library, there are one or two things that you need to add to the above command line:
You need to tell the linker to link with the library, this is done with e.g. the -l (lower case L) option:
$ gcc -Wall -g source1.c source2.c -o your_program_name -lthe_library
It's important to note that the_library is the base name of the library. If the library file is named libthe_library.so then only the_library part is needed, the linker will add the other parts automatically.
If the library is not in a standard location, then you need to tell the compiler and linker where the library file are. This is done with the -I (capital i) option to tell the preprocessor where the header files are, and the -L (capital l) where the linker files are.
Something like
$ gcc -Wall -g -Ilocation/of/headers source1.c source2.c -o your_program_name -Llocation/of/libraries -lthe_library
If you have more than a couple of source files, it's common to use so called makefiles that lists all source files, their dependencies, compiler and linker flags, and contain rules on how to build object files and link the final program. Such a makefile could look like
CFLAGS = -Wall -g
LDFLAGS = -g
SOURCES = source1.c source2.c
OBJECTS = $(SOURCES:.c=.o)
TARGET = your_program_name
.PHONY: all
all: $(TARGET)
$(TARGET): $(OBJECTS)
$(LD) $(LDFLAGS) $^ -o $#
%.o: %.c
$(CC) $(CFLAGS) $< -c -o $#
The above makefile should do just about the same as the previous command line. The big difference is that it's much easier to add more source files, add special rules for special files, and most importantly, the make program will handle dependencies so that if one source file haven't been modified since last build then it won't be compiled. The last bit will make big projects with many source files build much quicker when only one or a few source files has been modified.
How to learn more about the topics [...]
By going to your favorite search engine, and looking for those topics there. I also recommend e.g. Wikipedia.
Of course, if you use an Integrated Development Environment (a.k.a. an IDE) then you don't have to compile from the command line, or to make your own makefiles, the IDE will handle all that for you. It will also have dialogs for the project settings where you can enter include paths and library paths, and what libraries to link with.
Why I do I need to place the header file that contains only function prototypes?
So as to satisfy your compiler for declaration of those functions or declaration of classes. As C++ is static type checking language, they must know the type of objects which they will be using.
Where to place my own files and how to compile it?
You can place you code anywhere in you filesystem; only make sure to include .h files in includes path and lib while compiling. Usually you need to modify your path.
You can check about building on this link:
https://en.wikipedia.org/wiki/GNU_build_system
Check the README file that came with the code. It should tell you how to install it into the system properly. Usually there is an install build target which installs the resulting files into the proper directories.
The usual sequence of commands to build and install most products is:
$ ./configure
$ make
$ sudo make install

C++: get Boost working; questions on include path and linking library

I was trying to use Boost.Tokenizer library. In my PROG.cpp, I have the following:
#include <boost/tokenizer.hpp>
And my Makefile was initially something like
CXX = g++-4.8
CXXFLAGS = ## some irrelevant flags
LDFLAGS = ## some irrelevant flags
SOURCES = PROG.cpp
OBJECTS = $(SOURCES:.cpp=.o)
TARGETS = PROG
$(TARGETS) : $(OBJECTS)
$(CXX) $(CXXFLAGS) -o $# $^ $(LDFLAGS)
## other targets
It won't compile, since boost/tokenizer.hpp cannot be found:
fatal error: boost/tokenizer.hpp: No such file or directory
Then I manually added the boost include path to CXXFLAGS:
-I/opt/local/include/
(which is the path from MacPorts.)
Then I tried to include the Tokenizer library, but in /opt/local/lib/ I have libboost_atomic-mt.dylib, libboost_chrono-mt.dylib, etc., but nothing like tokenizer. I was rather confused at the time. I supposed that still wouldn't work since the library was not linked against. Surprisingly, the program built, linked, and ran perfectly.
So I'm really confused now. Here are some questions:
(1) I did not link against boost explicitly, so boost is treated like standard library by the linker?
(2) If boost is treated like standard, why the headers are not standard?
(3) Why there are libboost_atomic-mt.dylib, libboost_chrono-mt.dylib, etc. but not tokenizer? Which dynamic library does tokenizer belong to?
I'm not very familiar with g++ linking mechanism; speaking of boost, this is my very first program with boost. So I'd really appreciate detailed explanation. Thanks in advance!
For reference, this is what I extracted by gcc -print-search-dirs:
install: /usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/
programs: =/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/bin/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/bin/
libraries: =/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/lib/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/lib/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../:/lib/x86_64-apple-darwin12.3.0/4.8.0/:/lib/:/usr/lib/x86_64-apple-darwin12.3.0/4.8.0/:/usr/lib/
Most of the boost libraries, are just header files, if you look in the .hpp files you will not see just the declaration of the classes, like you would expect in a header file, but actually the entire implementation. This is why for 90% of the boost libraries, you don't need to worry about linking, only inclusion.
However for a few libraries, serialiser, a few others, there is just too much polluting code for the header inclusion method to be reasonable. I'm sure there is a better, more rigid definition about when the implementation is included in the header and when it isn't.
http://www.boost.org/boost-build2/doc/html/bbv2/faq/header-only-libraries.html
Here is another question about it:
Why are not all boost libraries header-only?
p.s.
Generally it is better to keep the boost library separate and in your makefile do something like:
For compilation:
CXXFLAGS += -I/path/to/boost/include
For Linking:
LDPATH += -L/path/to/boost/lib
This makes it easier to upgrade your boost version as you just have to change the path in one place.

How do I correctly link in static libraries with g++?

I have a rather complex build I'm trying to do, but I'll simplify it a little bit for this question. I have three c++ files (main.cpp file2.cpp and file3.cpp) that I am trying to compile and link against 3 static libs (libx.a liby.z libz.a) to produce an executable.
There are many dependencies involved.
All three c files are dependent on all 3 libs. libx is dependent on liby and libz. And finally, libx is also dependent on several callback functions contained in file2.cpp.
What command line would build this correctly? I have tried dozens of variations and nothing has satisfied the linker yet.
If it matters, the libs are pure c code compiled with gcc. Sources are c++ and I'm compiling/linking with g++. I have this working correctly as a visual studio project, and am trying to port to linux.
From your post:
g++ main.cpp file2.cpp file3.cpp -lx -ly -lz
However, if static linking is causing you problems, or you need to distribute any of the libs, then you may consider making them shared objects (.so files, commonly called DSOs). In that case, when you build libx.a, for example, compile all the sources to object files, and then combine them with
g++ -shared *.o -o libx.so -ly -lz
(this version assumes that liby.a and libz.a are stills static, and will be combined into libx.so
You may need to use extern "C" { } in your .cpp files to include the header for the C libs.
See Including C Headers in C++ in How To Mix C and C++.

Why doesn't exist a relink button in compilers?

I work in Linux with c++, using eclipse. But i have worked with Visual Studio too. They haven't got (or at least I donĀ“t know how to do it) a button to relink a project.
Example:
I have a big project (1), with hundreds of cpp. That project uses a small library (2) to do foo. If I change foo behavior, and compile it, generating a library, I need to clean the big proyect (1), re-compile, that links the external libraries (2) and works.
The problem is the big project doesn't change, but with hundreds of cpps, its compile time is about 5 min. 5 min is a small change in a second library.
Is possible to avoid this problem?
Thanks in advance
I suspect, but it is just a guess, your project is missing a dependency between your foo library and other deliverables in your project.
In this way, when you modify foo, the compiler does not know that it needs to recompile (as much as required by the change in foo) the rest of the project, and this forces you to manually clean and rebuild.
Usually a dependency is specified in a highly compiler-specific way, maybe this other post from S.O can help you, or just google "eclipse c++ dependency management".
You would generally use make with a makefile for this.
With this method, you can generate your own rules for building code, including bypassing compilation of lots of source files if you only thing needed is relinking.
For example, the makefile:
prog: main.o other.o makefile
gcc -o prog main.o other.o
main.o: main.c makefile
gcc -c -o main.o main.c
other.o: other.c makefile
gcc -c -o other.o other.c
would not recompile main.c if the only file you changed was other.c. It would simply compile other.c to make other.o, then link other.o and main.o together to create prog.
That's generally how it's done in the "command line" world. It's likely that it's also how it's done behind the curtains in many IDEs as well, just hidden from you.
What you'll need to find out is why the dependency checking is not working as expected. Without further information on how your project is set up, it's a little hard to be definitive.
Add your static library under the Linker-Settings->Miscellaneous->Other objects. Your executable will just be relinked, in case the library is newer. No compilation.