I am having difficulties with re-writting my C++ library into template form; main problem concerns re-designing the Makefile.
In the previous state, when it was non-template I had:
a header file lib.h in which (protected by include guards) we have the declarations of classes and overloaded operators. This file does not include any other libraries at all.
an implementation file lib.cpp in which, on top of the file, I include many headers from the standard library (cmath, iostream, etc) as well as I include the header file for this custom library: #include "lib.h"
a Makefile with commands for build/install the library:
CC = g++
CFLAGS = -O2
SRC = lib.cpp
HDR = $(SRC:.cpp=.h)
OBJ = $(SRC:.cpp=.o)
LIB = $(OBJ:.o=.a)
.PHONY: all install clean uninstall
# =========================================================
# Build
# =========================================================
all: $(LIB)
# Create an object file
$(OBJ):
$(CC) $(CFLAGS) -c -o $# $(SRC)
# Create a static library file
$(LIB): $(OBJ)
ar rcs $# $<
# =========================================================
# Install
# =========================================================
install: ~/lib/lib/$(LIB) ~/lib/include/$(HDR)
# Create top-level directory for the libraries
~/lib:
mkdir -p $#;
# Create top-level directory for the static library files
~/lib/lib:
mkdir -p $#;
# Create top-level directory for the headers
~/lib/include:
mkdir -p $#;
# Copy the library file into the right directory
~/lib/lib/$(LIB): $(LIB) ~/lib/lib
cp $< $#
# Copy the header file into the right directory
~/lib/include/$(HDR): $(HDR) ~/lib/include
cp $< $#
following the installation my CI (GH Actions) would compile a small test program, which includes my library header (#include <lib.h>) with the following command:
g++ -O0 -Wall --std=c++14 test.cpp -I ~/lib/include/ ~/lib/lib/lib.a -o test
This setup worked fine.
Problems emerge now, when I want to re-write my class to be template.
Following the information in this post I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc). With that change I needed to adjust the compilation process and do not provide my implementation file in the command line (as it is already included in the header and I have to avoid re-definition errors). That works fine. The core of the problem is now in the Makefile, in the command to build an object file. As I try to compile the implementation file of my library, it includes the header and the header includes the implementation again... I red about the problem in this port and they suggest to remove the header inclusion from the implementation file. So I did that, I commented the #include "lib.h" and tried to run:
g++ -O2 -c -o lib.o lib.cpp
And I end up with a lot of error: use of undeclared identifier errors...
How can I build the library properly with make? My constraint is that the library stays in two separate files: header and implementation. What I want in the end is to be able to #include <lib.h> in my further programs.
Is it even possible to create an archived object file? What about shared object library (.so)
I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc).
I can't say I think much of the advice you followed there. Code that is intended to be used as or in a header should be named appropriately (.h, .hpp, or whatever convention your project follows), and code that is suitable for direct compilation pretty much never belongs in a header. Perhaps your conversion involves changing everything of the latter kind into to former kind, so maybe you want to rename lib.cpp to lib_impl.h, or something similar, and skip trying to compile it at all. Maybe.
Note well, however, that if you name and structure the implementation code as a header, then it needs its own include guards. Note also that it must not, then, contain any external, non-template functions. If it did, then no two separate translation units contributing to the same program could both include the header, as that would result in duplicate function definitions, include guards notwithstanding. And if there are no external, non-template functions (and no external object definitions) then it is pointless to try to compile to an object file, as there would be no accessible entry point to any function within.
How can I build the library properly with make?
This isn't really about make in particular. That just provides automation. The issue is with the structure of the library itself and your expectations.
My constraint is
that the library stays in two separate files: header and
implementation.
This constraint makes sense only if the implementation contains any instantiated templates for external objects or functions, or any external non-template functions or objects. These are the things that would contribute to a buildable object file that you could directly or indirectly link to applications. If converting to a template library means that there are no longer any such entities in your library then the constraint is arbitrary, as the converted result is a header-only library. Nevertheless, you can split up your headers any way you like, as long as each resulting one is structured as a header, with only contents appropriate for a header, and with its own inclusion guards.
On the other hand, if your converted implementation code does contain any of those things then they must not be #included into any header, as discussed above.
What I want in the end is to be able to #include <lib.h> in my further programs.
If the converted implementation code is suitable for use as or in a header, then you are already there, but it would be much better style to rename your lib.cpp as a header. If you want to be able to include that header directly into code other than the main library header then it needs its own inclusion guards -- that will take care of the duplicate declarations. Do note, however, that those errors arise from the fact that you have a circular dependency, and that's a strong sign that you ought to refactor. Such a refactoring would involve moving enough code from lib.cpp to lib.h that the circular dependency can be removed (one of the files would no longer #include the other).
Whatever implementation code is not suitable for use in a header obviously must not be included into the header. If any such code remains in the converted library then perhaps you keep that in lib.cpp, and move the rest into lib.h
Is it even possible to create an archived object file? What about shared object library (.so)
Templates cannot be compiled. They are templates for code that can be compiled: their instantiations. If your conversion leaves nothing else then no, you cannot usefully create an object file or a shared library. And you don't need to do, for this is a header-only library.
Related
I have a conceptual question regarding makefile dependencies and this is because I see inconsistency online about this.
Let's say I have the following files:
main.cpp uses-> my_math.cpp and my_strings.cpp
my_math.cpp uses-> my_math.h
my_strings.cpp uses-> my_strings.h
If I have a makefile, with the general outlay of:
program: $(all_objs)
g++ $(all_objs) -o program
main.o: ...
.......
my_math.o: ...
.......
my_strings.o: ...
.......
I don't know what should go into each dependency. Like, math.o #includes my_math.h and my_strings.h, does that mean that main.cpp needs to recompile if I change my_math.h? But why? It uses it like a library, right? It doesn't need to recompile main.cpp or does it?
As in, should the result for main.o be:
1) main.o: main.cpp
gcc -c main.cpp
2) main.o: main.cpp my_strings.cpp my_strings.h my_math.cpp my_math.h
gcc -c main.cpp
3) main.o: main.cpp my_strings.cpp my_strings.h my_math.cpp my_math.h
gcc -c main.cpp my_strings.cpp my_math.cpp
I'm a bit lost on dependencies and how linking works.
Any help would be appreciated! Thank you!
The dependencies are everything whose change requires recompiling the source code. That includes not only your #include-d headers, but also the indirectly included system headers, and even (in principle) the compiler and build chain itself (when you upgrade your C++ compiler, you should recompile all your software). If some of your C++ code is generated from some source (e.g. by tools like GNU bison or Qt moc, or by your own script), both the sources and the generating tools are dependencies. Read also about package managers.
Practically speaking, the GCC compiler is able to output most make dependencies, notably with -M and related processor options. Read also about auto dependencies generation. See also this.
(in practice, you generally don't code in your Makefile some explicit dependency on the compiler itself; but you should not forget to make clean when the compiler has been upgraded)
Unless your main.cpp is including my_strings.cpp (which is not conventional and is very bad taste), your make rule won't have a dependency from my_strings.cpp to main.o. But probably your main.cpp is #include-ing (directly or indirectly) my_strings.h so main.o should depend not only on main.cpp but also on my_strings.h
As a rule of thumb, your object file my_strings.o depends on the source file my_strings.cpp and all the header files which are directly or indirectly #include-d in it. Your main program executable depends on all its object files and the libraries you are linking into it. Order of program arguments to g++ matters a lot.
It uses it like a library, right?
From what you are showing, you don't have any own libraries (but you probably use the standard C++ library, and perhaps some other system libraries). On Linux these are lib*.a files (static libraries) or lib*.so files (shared libraries). A library is an organized agglomeration of object code -and sometimes other resources.
I'm a bit lost on dependencies and how linking works.
Understand the difference between source code files, object files (they contain relocation information) and executables (on Linux, object files and executable files and shared libraries are using the ELF format). Read also about the role of compilers, linkers (the g++ program can run both) & build automation (for which you are using make).
Read Program Library HowTo and much more about translation units and linkers (& name mangling), notably Levine's book on Linkers & loaders.
See also this & that & this (examples about Makefile for C++ programs).
BTW, you should use g++ (not gcc) when compiling C++ code. There are significant differences (even if gcc is sometimes able to compile C++ or Fortran code, you'll mostly use gcc to compile C code). And (assuming you use specifically GNU make) your Makefile should mention $(CXX) (not g++). You need to understand the builtin rules of make (run once make -p to get them) and you'll better take advantage of them (e.g. use $(COMPILE.cc) or $(COMPILE.cpp) etc...). You certainly should pass -Wall -Wextra (to get all warnings, and even more), and -g (to get debugging information) to g++. Practically speaking, you should set your CXXFLAGS variable in your Makefile.
Take time to carefully read GNU make documentation and Invoking GCC.
Look into the Makefile-s of existing free software projects. For various reasons, some projects are generating their Makefile-s with tools like autoconf or cmake. But most simple projects don't need that generality, and you should be able to write your own Makefile for your C++ projects. Of course, take inspiration from existing code.
If you have
main.cpp uses-> my_math.cpp and my_strings.cpp
my_math.cpp uses-> my_math.h
my_strings.cpp uses-> my_strings.h
The purpose of Make is to maintain dependency between modules in two different ways, by building .o files and by linking .o files.
you can picture it as a dependency tree where main.o is the root
main.o
/ \
my_math.o my_strings.o
for each .o there is also a dependency tree with regard to source files e.g.
main.o my_math.o my_strings.o
/ \ / \ / \
main.cpp main.h my_math.cpp my_math.h my_strings.cpp my_strings.h
So when make builds, it sets up a dependency tree with main.o at its root and then tries to build all .o files needed for main. When all .o files have been built they are linked.
By following the dependency tree Make ensures that main will be linked/built when one of the dependent modules is changed.
However if you have used something like say a constant from one of the included headers #define MAXSTRING 32 you are no longer merely dependent on the .o file, you are then dependent on the header content so you need to make sure that main.o is built if the header is changed since linking is then not enough so you add the .h in the dependency
main.o
/ | \
my_math.o my_strings.o my_strings.h
Of course there are ways to make the header more robust to avoid that dependency but is another question.
Your cpp files do not depend on other cpp files in terms of compilation. Simple cpp file should only depend on h files.
In your question you say main.cpp depends on my_math.cpp and my_strings.cpp, but I think it is not true. I can guess you have #includes there and these are your dependecies.
Generally speaking dependencies for cpp files are all #included h files.
Usually there are no dependencies between cpp files. You just generate o files by compiling them. Then your final binary depends on all o files.
The background about this question is: my project(C++ language) contains too many files, which including boost, thrift, zookeeper, etc. Now the compilation duration takes too long.
As you know, Visual Studio supports precompiled headers, so as GCC. Because I use automake to manager the Make procedure, so What I want to ask for is whether automake supports precompiled headers? How can I write automake files if so?
Waiting and thranks for your answers.
The whole thing is not about automake, but rather about writing makefiles.
Actually, you can try the .gch, but there're restrictons:
write one common header to include everything (like stdafx.h)
It should be the 1st include in all your sources
use same CFLAGS to compile all your sources
It's a new functionality. You'd want to detect it in your configure.ac
Write a rule in your Makefile.am, name it like stdafx.gch:. Make it empty, if gch is not supported:
stdafx.gch: stdafx.h
$(OPTIONAL_COMPILE_GCH)
.PHONY: $(OPTIONAL_STDAFX_GCH)
Make your _OBJECTS depend on stdafx.gch:
$(foo_SOURCES:.cpp=.$(OBJEXT)): stdafx.gch
# or (is it documented var?)
$(foo_OBJECTS)): stdafx.gch
Youc can use the documented CXXCOMPILE command to be sure that all CXXFLAGS are the same.
Detect in your configure.ac:
if ...; then
[OPTIONAL_COMPILE_GCH='$(CXXCOMPILE) -c -o $# $<']
[OPTIONAL_STDAFX_GCH=]
else
[OPTIONAL_COMPILE_GCH=]
[OPTIONAL_STDAFX_GCH='stdafx.gch']
fi
AC_SUBST(OPTIONAL_COMPILE_GCH)
AC_SUBST(OPTIONAL_STDAFX_GCH)
Update
You want your .gch to be recompiled, when any header file indirectly used by stdafx.h is modified.
Then you could add stdafx.cpp that does nothing, but includes stdagx.h and make .gch depend on stdafx.o. This way the dependency tracking would be managed by automake, but there's a problem:
stdafx.o itself could make use of .gch to compile faster, but if we add such dependency, it would be circular. It's not easy for me to find a solution.
Here's an example that uses status files to solve this: https://gist.github.com/basinilya/e00ea0055f74092b4790
Ideally, we would override the compilation command for stdafx.o, so it first creates .gch and then runs the standard automake compilation, but automake does not use $(CXXCOMPILE) as is. It creates a complex recipe from it (and it depends on automake version):
Update 2
Another solution is to use gcc dependency tracking directly:
stdafx.h.gch: stdafx.h
g++ -MT stdafx.h.gch -MD -MP -MF .deps/stdafx.Tpo -c -o stdafx.h.gch stdafx.h
mv .deps/stdafx.Tpo .deps/stdafx.Po
-include .deps/stdafx.Po
By default, if precompiled header is used, gcc -MD will not put the list of header files into the generated dependencies file, just .gch. There's an option to fix this : -fpch-deps (see bug), but perhaps the default behavior is not that bad: if only the .gch depends on the headers, make will have less work to do.
I'm new to C and wanted to use a library (MLT Multimedia Framework)
I've built it and it produced the following directories: include lib share
Inside lib there are .so .a .la files
Inside include there are .h files
Now, I'm instructed to do this:
#include <framework/mlt.h> which is inside include/mlt/framework/
Questions:
Why I do I need to place the header file that contains only function prototypes? Where are the real functions then? are they linked someway to the ones included in lib directory?
Where to place my own files and how to compile it?
How to learn more about the topics:
Dynamic/Static libraries
Building / making / installing
How to use any C library
If you don't have the function prototypes, how would the compiler know what functions exist in the library? Short answer is: It doesn't. Longer answer: The compiler doesn't care about library files, static (files ending in .a) or shared (files ending in .so), all it cares about is the current translation unit. It's up to the linker to handle resolving undefined references.
When you use libraries, you include the header files that contain the needed declarations (structures, classes, types, function prototypes) into the source file. The source file plus all included header files forms the translation unit that the compiler uses to generate code. If there are undefined references (for example a call to a function in the library) the compiler adds special information about that to the generated object file. The linker then looks through all object files, and if it finds an unresolved reference it tries to find it in the other object files and the provided libraries. If all definitions are resolved the linker generates the final executable, otherwise it will report the unresolved definitions as errors.
To answer your other questions:
Where to place my own files and how to compile it?
This is two questions, the answer to the first one (about placement of your files) is that it doesn't really matter. For small project with only a few source and header files, it's common to place all files in a common project directory.
The second question, about compiling, there are different ways to do it too. If there are only one or two source files you could use the compiler frontend (e.g. gcc) to compile and link and generate your executable all in one go:
$ gcc -Wall -g source1.c source2.c -o your_program_name
The above command takes two source files, compiles and links them into the program your_program_name.
If you need to use a library, there are one or two things that you need to add to the above command line:
You need to tell the linker to link with the library, this is done with e.g. the -l (lower case L) option:
$ gcc -Wall -g source1.c source2.c -o your_program_name -lthe_library
It's important to note that the_library is the base name of the library. If the library file is named libthe_library.so then only the_library part is needed, the linker will add the other parts automatically.
If the library is not in a standard location, then you need to tell the compiler and linker where the library file are. This is done with the -I (capital i) option to tell the preprocessor where the header files are, and the -L (capital l) where the linker files are.
Something like
$ gcc -Wall -g -Ilocation/of/headers source1.c source2.c -o your_program_name -Llocation/of/libraries -lthe_library
If you have more than a couple of source files, it's common to use so called makefiles that lists all source files, their dependencies, compiler and linker flags, and contain rules on how to build object files and link the final program. Such a makefile could look like
CFLAGS = -Wall -g
LDFLAGS = -g
SOURCES = source1.c source2.c
OBJECTS = $(SOURCES:.c=.o)
TARGET = your_program_name
.PHONY: all
all: $(TARGET)
$(TARGET): $(OBJECTS)
$(LD) $(LDFLAGS) $^ -o $#
%.o: %.c
$(CC) $(CFLAGS) $< -c -o $#
The above makefile should do just about the same as the previous command line. The big difference is that it's much easier to add more source files, add special rules for special files, and most importantly, the make program will handle dependencies so that if one source file haven't been modified since last build then it won't be compiled. The last bit will make big projects with many source files build much quicker when only one or a few source files has been modified.
How to learn more about the topics [...]
By going to your favorite search engine, and looking for those topics there. I also recommend e.g. Wikipedia.
Of course, if you use an Integrated Development Environment (a.k.a. an IDE) then you don't have to compile from the command line, or to make your own makefiles, the IDE will handle all that for you. It will also have dialogs for the project settings where you can enter include paths and library paths, and what libraries to link with.
Why I do I need to place the header file that contains only function prototypes?
So as to satisfy your compiler for declaration of those functions or declaration of classes. As C++ is static type checking language, they must know the type of objects which they will be using.
Where to place my own files and how to compile it?
You can place you code anywhere in you filesystem; only make sure to include .h files in includes path and lib while compiling. Usually you need to modify your path.
You can check about building on this link:
https://en.wikipedia.org/wiki/GNU_build_system
Check the README file that came with the code. It should tell you how to install it into the system properly. Usually there is an install build target which installs the resulting files into the proper directories.
The usual sequence of commands to build and install most products is:
$ ./configure
$ make
$ sudo make install
I was trying to use Boost.Tokenizer library. In my PROG.cpp, I have the following:
#include <boost/tokenizer.hpp>
And my Makefile was initially something like
CXX = g++-4.8
CXXFLAGS = ## some irrelevant flags
LDFLAGS = ## some irrelevant flags
SOURCES = PROG.cpp
OBJECTS = $(SOURCES:.cpp=.o)
TARGETS = PROG
$(TARGETS) : $(OBJECTS)
$(CXX) $(CXXFLAGS) -o $# $^ $(LDFLAGS)
## other targets
It won't compile, since boost/tokenizer.hpp cannot be found:
fatal error: boost/tokenizer.hpp: No such file or directory
Then I manually added the boost include path to CXXFLAGS:
-I/opt/local/include/
(which is the path from MacPorts.)
Then I tried to include the Tokenizer library, but in /opt/local/lib/ I have libboost_atomic-mt.dylib, libboost_chrono-mt.dylib, etc., but nothing like tokenizer. I was rather confused at the time. I supposed that still wouldn't work since the library was not linked against. Surprisingly, the program built, linked, and ran perfectly.
So I'm really confused now. Here are some questions:
(1) I did not link against boost explicitly, so boost is treated like standard library by the linker?
(2) If boost is treated like standard, why the headers are not standard?
(3) Why there are libboost_atomic-mt.dylib, libboost_chrono-mt.dylib, etc. but not tokenizer? Which dynamic library does tokenizer belong to?
I'm not very familiar with g++ linking mechanism; speaking of boost, this is my very first program with boost. So I'd really appreciate detailed explanation. Thanks in advance!
For reference, this is what I extracted by gcc -print-search-dirs:
install: /usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/
programs: =/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/bin/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/bin/
libraries: =/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/lib/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/lib/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../:/lib/x86_64-apple-darwin12.3.0/4.8.0/:/lib/:/usr/lib/x86_64-apple-darwin12.3.0/4.8.0/:/usr/lib/
Most of the boost libraries, are just header files, if you look in the .hpp files you will not see just the declaration of the classes, like you would expect in a header file, but actually the entire implementation. This is why for 90% of the boost libraries, you don't need to worry about linking, only inclusion.
However for a few libraries, serialiser, a few others, there is just too much polluting code for the header inclusion method to be reasonable. I'm sure there is a better, more rigid definition about when the implementation is included in the header and when it isn't.
http://www.boost.org/boost-build2/doc/html/bbv2/faq/header-only-libraries.html
Here is another question about it:
Why are not all boost libraries header-only?
p.s.
Generally it is better to keep the boost library separate and in your makefile do something like:
For compilation:
CXXFLAGS += -I/path/to/boost/include
For Linking:
LDPATH += -L/path/to/boost/lib
This makes it easier to upgrade your boost version as you just have to change the path in one place.
I'm trying to compile a third-party library, but g++ is complaining about the following line:
typedef boost::shared_ptr<MessageConsumer> MessageConsumerPtr;
The strange thing is, there is no #include directive in the file - and it is clearly supposed to be this way; there are about 60 files with the same (or very similar) issues. Clearly if there was an #include directive referencing the relevant boost header this would compile cleanly.
My question is: how can I get g++ to somehow automagically find the relevant symbol (in all instances of this issue, it is a namespace that can't be found - usually std:: or boost::) by either automatically processing the relevant header (or some other mechanism).
Thanks.
Edit
My current g++ call looks like:
g++ -fPIC -O3 -DUSING_PCH -D_REENTRANT -I/usr/include/boost -I./ -c MessageInterpreter.cpp -o MessageInterpreter.o
You can use the -include command line option:
g++ -include boost/shared_ptr.hpp ...
From the man page:
-include file
Process file as if "#include "file"" appeared as the first line of
the primary source file. However, the first directory searched for
file is the preprocessor's working directory instead of the
directory containing the main source file. If not found there, it
is searched for in the remainder of the "#include "..."" search
chain as normal.
Create your own wrapper .h file that includes the boost .h and then the broken .h .
Or (very fragile) ensure that you precede every use of the broken .h with boost .h .
Perhaps the third-party library is designed in such a way that you should always include a certain "main" header file in order to get the dependencies right.
Otherwise, you can add #include <boost/shared_ptr.hpp> before including the third-party header file that is giving the error message.