I coded a program importing and making use of many headers from the boost library. After that the compilation time went from 1-2 seconds to 30s+. Since I am always importing the same library I was wondering if there was a way to compile them once and for all to speed up the following compilations.
I am very unfamiliar with shared/static libraries and could not find a tutorial answering my question.
I do not mind have a much larger executable in case this is the price to pay.
If you include boost headers in one of your program's headers, and all / many of your program's .cpp files include that header, then the boost headers end up getting included in all of your .cpp files, and get compiled once for each one of them.
To avoid this, you can try to only include boost headers in (one) or a few .cpp files in your project.
You can also use PIMPL idiom, also known as "compilation firewall". The idea is that you expose only an interface in the header that your program uses, and if the implementation requires boost things, then that appears in the .cpp file only so you dont end up including boost everywhere.
Note that header-only libraries don't really have to do with shared vs. static. With shared / static libraries, you have object code of some kind which was obtained by compiling the libraries in advance. With header-only libraries, what you are importing is just template definitions into your code, and your compiler makes use of them. It's closer in spirit to static than to shared linking, but it's not really either.
Related
I couldn't really find anything on Google or SO about this. I don't know if it's because it's uncommon, or because I am using the wrong terms to search. I read this question here, but it didn't really answer my question.
So what I am doing is trying to build a library on top of SDL for C++. Now what I can't seem to figure out is how to make projects that use my library be totally independent from SDLs files except the dll.
When I make my library, I link with SDL2.lib, and SDL2main.lib. I include all of SDLs header files into my libraries files. When I build, it generates my library file; GGL.lib.
But when I want to test my library in another project, I have to include all of SDLs header files, because in my project's Window.h, it includes SDL.h. I am wondering if there is any way for me to make my libraries header files independent from SDL except for the dll.
You obviously need to avoid including SDL header files in header files of your library. Include them only in source files, where possible. This removes redundant dependencies and speeds up compilation process. But you want complete independence, so if you can't remove one #include directive, you failed.
If you need to declare a pointer of an SDL type in your header (like one usually does with SDL), forward declare it. That type will however remain incomplete to the users of your library. In case your library is object-oriented, there's Pimpl Idiom, which is based on the same principle - this hides everything.
Let's say I have 10 *.hpp and *.cpp files that I need to compile a code. I know that I will need those same files for many different codes. Can I create a "package" with those files that would allow me to simply write:
#include <mypackage>
instead of:
#include "file1.hpp"
#include "file2.hpp"
...
#include "file10.hpp"
I wouldn't then need to write a makefile every time I need this "package".
To be more precise, I use Linux.
A collection of CPP sources (H files and CPP files) can be compiled together in to a "library," which can then be used in other programs and libraries. The specifics of how to do this are platform- and toolchain-specific, so I leave it to you to discover the details. However, I'll provide a couple links that you can have a read of:
Creating a shared and static library with the gnu compiler [gcc]
Walkthrough: Creating and Using a Dynamic Link Library (C++)
Libraries can be seperated in to two types: source code libraries, and binary libraries. There can also be hybrids of these two types -- a library can be both a source and binary library. Source code libraries are simply that: a collection of code distributed as just source code; typically header files. Most of the Boost libraries are of this type. Binary libraries are compiled in to a package that is runtime-loadable by a client program.
Even in the case of binary libraries (and obviously in the case of source libraries), a header file (or multiple header files) must be provided to the user of the library. This tells the compiler of the client program what functions etc to look for in the library. What is often done by library writers is a single, master header file is composed with declarations of everything that is exported by the library, and the client will #include that header. Later, in the case of binary libraries, the client program will "link" to the library, and this resolves all the names mentioned in the header to executable addresses.
When composing the client-side header file, keep complexity in mind. There may be many cases where some of your clients only want to use some few parts of your library. If you compose one master header file that includes everything from your library, your clients compilation times will be needlessly increased.
A common way of dealing with this problem is to provide individual header files for correlated parts of your library. If you think of all of Boost a single library, then Boost is an example of this. Boost is an enormous library, but if all you want is the regex functionality, you can only #include the regex-related header(s) to get that functionality. You don't have to include all of Boost if all you want is the regex stuff.
Under both Windows and Linux, binary libraries can be further subdivided in to two types: dynamic and static. In the case of static libraries, the code of the library is actually "imported" (for lack of a better term) in to the executable of the client program. A static library is distributed by you, but this is only needed by the client during the compilation step. This is handy when you do not want to force your client to have to distribute additional files with their program. It also helps to avoid Dependancy Hell. A Dynamic library, on the other hand, is not "imported" in to the client program directly, buy dynamically loaded by the client program when it executes. This both reduces the size of the client program and potentially the disc footprint in cases where multiple programs use the same dynamic library, but the library binary must be distributed & installed with the client program.
On Linux:
g++ FLAGS -shared -Wl,-soname,libLIBNAME.so.1 -o libLIBNAME.VERSION OBJECT_FILES
where
FLAGS: typical flags (e.g., -g, -Wall, -Wextra, etc.)
LIBNAME: name of your library
OBJECT_FILES: objects files resulting from compiling cpp files
VERSION: version of your library
Assuming your "file1.hpp" and "file2.hpp" etc are closely related and (nearly) always used together, then making one "mypacakge.h" that contains the includes of the other components is a good idea (it doesn't in and of itself make it into a library - that is a different process altogether).
If they are NOT closely related and/or used together, then you shouldn't have such a "mega include", because it just drags in a bunch of things that aren't needed.
To make a library involves building your code once, and either generating a .lib file or a shared librar (.dll or .so file). The exact steps to do this depends on what system you are using, and it's a little too complicated for me to explain here.
Edit: To explain further: All of the C++ library is actually one library file or shared library file [along with a number of header files that contain some of the code and the declarations needed to use the code in the library]. But you include <iostream> and <vector> separately - it would become pretty awful to include EVERYTHING from all the different C++ library headers in one <allcpplibrary>, even if it was a lot less typing involved. It is split into sections that do one thing per headerfile. So you get a "complete" set from one header file, but not a too much other things you don't actually need.
Yes and no.
You can write an include-all header so that #include "myLib.h" is sufficient, because you include all those headers through the single header. However, that does not mean that the single include is enough to have the content of the 10 '.cpp' files linked to your project automagically. You will have to compile them into a library and link that single library (instead of all the object files) to the projects that use "myLib.h". Library binaries come as static and dynamic libraries, the files are typically named .lib and .dll (windows) and .a and .so (linux) for static and dynamic libraries, respectively.
How to build and link such libraries depends on your build system, you might want to loke those terms up on the net.
One alternative is to get rid of the .cpp files by defininig all the functions in the headers. That way you won't have to link the additional library, but it comes at the cost of increased build times, because the compiler will have to process all those functions every time you include the header directly or indirectly into one of your translation units.
If a client needs all ten headers to actually make use of your "package" (library), that's pretty bad interface design.
If a client needs only some headers, depending on which parts of your library are being used, let the client include the appropriate headers, so only a minimal set of identifiers are introduced. This helps scope, modularization, and compilation times.
If all else fails, you can make an "interface header" for external use, which is different from the ones you use internally for actually compiling your library. This would be the one that gets installed, and consists of the necessary contents from the other headers. (I still don't think you would need everything from every header in your lib.)
I would discourage Salgar's solution. You either have individual headers, or a monolithic one. Providing individual headers plus a central one that simply includes the others strikes me as pretty poor layout.
What I do not understand is inhowfar Makefiles play into this. Header dependencies should be resolved automatically by your Makefile / build system, i.e. it shouldn't matter here how your header files are layed out.
simply all you'd have to do is create a .h or .hpp file that has
#ifndef MAIN_LIB_H
#define MAIN_LIB_H
#include "file1.hpp"
#include "file2.hpp"
#include "file3.hpp"
...
#include "file10.hpp"
#endif
make the file called whatever I would choose main_lib.h because of the ifndef, and just
#include "DIRECTORY PATH IF THERE IS ONE/main_lib.h"
in the main file. No need for anything else if you were using visual studios. Just build then press CTRL + F5.
We can include < iostream > and we do not care about its cpp file, but why can't we do the same for our own classes ?
So if my project uses 50 custom classes, not only do I have to have 50 includes, but also have to compile/link 50 cpp files along with them (and clutter the project tree).
Q: Is there any way to use custom headers the same way we use standard libraries ?
In other words is there a kosher way so that we do not have to add all those cpp files in the project. I want to only include ClassSnake.hpp which in turn knows where to find ClassSnake.cpp which links to ClassVector.hpp which knows how to find ClassVector.cpp ... all in an automatic daisy chain without me having to explicitly add those cpp files in my project tree.
Edit: I am not worried so much about the cpp files recompiling. My issue is with having to remember which class internally links to which other class, so that I can properly include all those hidden cpp files in the project tree ... and clutter the tree.
Not really.
What you're missing is that your compiler toolchain has already compiled the bits <iostream> needs that aren't in the header.
Your compiler (linker really) just implicitly links this code in without you having to specify it.
If you want to clean up your project tree a bit, you could create other projects that are libraries of code for one main project to use.
Headers do not (normally) provide implementations of things (functions, classes), they must be implemented somewhere if you are going to use them.
When you include your own header, you include your own sources in order to provide the implementation. Straight forward enough there.
When you include a standard header (such as iostream), the implementation is in the libraries you include (either implicitly because the compiler just does it, or explicitly through compiler/linker options).
As an extention to Collin's answer, you could always offload "shared" code to a shared library, and then reference the header files and lib files in your other projects. The lib files will only come in to play at the linker stage, as all those other pesky .cpp files have already been compiled.
If this is is just a self-enclosed project with no other commonality, you're just going to have to suck up the fact you have to provide an implementation :)
First of all if you use a system such as make then it will identify that the .cpp file has not changed and therefore the compiler does not have to reconstruct the object file.
You can also create your own static/shared library. The method to do this depends on the platform. If you go down this avenue then all you need is the header file along with the library.
Please Google on how to construct how to make a library for you particular platform.
Actually, if you have a decent build process cpp files that have not changed will not be compiled again. They only have to be linked. If you don't want that either you need to create your own libraries. It can be done, is just a bit more involved.
Edit: This question might help you if you want to create your own library.
So answer to edited question: Yes, you can avoid having all those cpp files in the project but only if you don't want to change them. In this case you can just create a static or dynamic library and you will only need the symbols for linking. In that case you would create another project to compile everything into such a library.
STL code like "iostream" is made of templates no code is actually generated until instances of the templates are created.
I'm new to C++. I just downloaded the Boost libraries to study. I wanted to look into some implementation details, so I looked for .cpp files. To my surprise, I haven't found any so far.
There seem only .hpp files out there. Where are the .cpp files?
From the Boost documentation:
Most Boost libraries are header-only: they consist entirely of header
files containing templates and inline functions, and require no
separately-compiled library binaries or special treatment when
linking.
See that link for the list of libraries that are not header-only and must be built separately. For those libraries, the .cpp files are in the /libs directory of the Boost distribution. If you got the precompiled package, you'll instead find the already-compiled .lib files in the /lib directory.
The .hpp files are the headers that you must include in your code in order to use Boost classes. Many Boost libraries are header-only; all of the implementations are in the .hpp files. For those that do have source, you only see the compiled versions as .lib files.
If you download a source distribution of Boost, it should have several subdirectories:
boost: contains .hpp headers
lib: contains .lib files (compiled implementation details)
libs: the source of those implementation details
Because many (but not all) of the libraries are implemented using templates, and must therefore be placed within header files.
A lot of the Boost library are purely template. In the previous standard of C++ there was already the keyword export to allow the developer to separate the implementation from the interface.
The sad truth was that the keyword never worked completely (difficult to implement from the compiler vendor point of view and difficult to use it right for the developer). One way to fix the problem was provide interface and implementation in a header file and avoid the implementation file. By the way, there are several Boost libraries that you need to compile and link in order to use it, and I bet you will find implementation files in those libraries.
For those which aren't header-only, the source files can be found inside the libs sub directory.
I believe the majority of the Boost libraries are implemented in the actual header files only, as previous posters mentioned. As was also mentioned, compiled implementation code will be included as separate library files when separate from the header files.
You mentioned being new to C++, so I think it's worth mentioning that this type of library distribution is not particular to Boost. Other third party libraries and APIs you use will likely be structured in the same way; you will find packages of header files and library files only, with no .c, .cxx, .cpp, etc files. This is done for a number of reasons, including to hide the implementation of the library functionality, and to allow shared libraries to be loaded into memory once each.
This article might help clarify things for you:
http://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/
You are probably looking at precompiled package, where cpp files are available in shape of libraries, not source. If you actually grab the source you will find some cpp files.
I know precompiled headers are used for speeding up compilations, but are there any do's and don'ts to what files I should include in them? For example, I have a project that uses a lot of boost libs so I just include the boost header files in stdafx.h (I'm using VS2008). Should I include every standard header file in them, too? Will this increase the size of my executeable even if I, for example, include <vector> but never use std::vector? Is it a bad idea to include my own project's header files in stdafx.h?
Generally speaking, every header file that you use across the application and that doesn't change often should go into the precompiled header file. This will speed up compilation because the precompiled header file gets compiled only once.
If you add a header file which changes often, you'll miss the point of the precompiled header file, because this often-changing header file will cause your whole project to recompile, possibly unnecessarily.
Specifically, defines a template class, so if you won't use std::vector, the overhead will not be big. However, I would advise against adding header files - however standard and generic - if you don't really need them. There IS some overhead to the compilation time, the binary size, and it could cause conflicts later in the project, so why add something if you don't really need it?
Pre-compiled headers don't affect the size of your executable, only the compilation speed. Since they are pre-compiled, they don't have to be re-compiled all the time. Windows.h is the primary beneficiary of this feature.
It's a good idea to include the c++ standard header-files and the boost library headers and any other headers from third party libraries that you frequently use. This will not affect the size of your executable.
However, you should not include headers from your own project, since the whole project needs to be rebuild whenever you make changes in these headers.