I'm working on a code base of template classes. It's header-only (no .cpp files). I would like to hide implementation and provide a shared library plus a couple of headers containing only declaration. Unfortunately sounds like doesn't make a sense. Since there is no compiled code, so what will be in such a shared lib? Trying to remove definitions from headers after compiling, causes undefined references. Is there a way to force compiler to ship objects in dll or shared library without having to explicitly instantiate template classes?
No, there isn't and wont be a way to do that for the foreseeable future. The only way to provide template C++ code is as header files only. Modules might change that, but that is unlikely to happen before your library is finished.
Something you can try is to split into implementation and explicitly instantiate all possible use cases. Then the library you ship wont work with any other types then the instantiated ones and would significantly reduce the benefit templates bring.
Template implementations need to be known at compile time. That's why you can't separate implementation from declaration. So if you want to have the advantages of templates, there is no way around passing your header(s).
Related
We are creating a set of libraries with a public API which is to be used by different third parties. Some of the libraries are pure C so obviously they have a C styled header with functions and struct definitions and the corresponding library. They are ok.
Some of the libraries are written with the usage of a moderately complex C++ (targeting older compilers), so there we have implemented some form of the famous pimpl idiom. This is ok too.
On the other end a significant part of the header files is C++ using heavily templated code. Knowing Why can templates only be implemented in the header file? but also not willing to disclose too much implementation details to eyes who are not supposed to see them we have heavily refactored them to exclude as much internal details as possible and having only the really necessary bits... and there is still a significant amount of code left.
So it puzzles me: Is there a preferred way of distributing header files which largely contain templates? What good practices, best approaches and tips and tricks are there?
Look at your C++ compiler's header files, for an inspiration. The standard C++ library is full of templates, and you will generally find all the template code in the headers.
Having said that, if particular templates are meant to be used with a small number of possible classes (or values) as template parameters, you do have an option of explicitly instantiating templates inside the library itself, leaving just the bare template declarations visible in the header files.
Using a simpler pre-C++11 scenario as an example, a C++ library will typically provide a std::basic_string implementation for only a std::basic_string<char> and std::basic_string<wchar_t>; and leave a bunch of template code inside the library itself, with just a bare std::basic_string template declaration visible in the header files.
Obviously template libraries need to be header only, but for non-templates, when should you make things header-only?
If you think your non-template library could be header-only, consider dividing it into two files anyway, then providing a third file that includes both the .h and the .cpp (with an include guard).
Then anyone who uses your library in a lot of different TUs, and suspects that this might be costing a lot of compile time, can easily make the change to test it.
Once you know users have the option which way to use the library, the answer probably becomes "offer that option whenever you possibly can". So pretty much any time that including it from multiple TUs wouldn't violate the ODR. For instance, if your non-static free functions refer to static globals, then you're out of luck, since the different definitions of that function in different TUs would refer to different objects by the same name, which is an ODR-violation.
You could follow Boost.Asio lead.
They simply provide the two versions of the libraries: header-only and header + library.
They do it with a single macro to be defined (or not) before including their headers. I think the default (if not defined) is to use the header-only version.
See Optional Separate Compilation.
Note how they neatly provide a single source file to be compiled that define everything or the option to link against a dynamically loaded library.
Template libraries need not to be header-only: implementations might well contain some pieces independent of template parameters, and for some reasons (e.g. less code size) separated into a special binary.
I cannot imagine a case where a non-template library really must be header-only. However sometimes it might be reasonable performance-wise to allow inlining of all the code. An example can be a library of wrappers around platform-specific interfaces, e.g. for things like synchronization primitives, thread-local storage, platform- and compiler-specific implementation of atomic operations etc.
Without templates, you'd have actual definitions in the headers. That means that if two files include your header, you'd get multiple definitions and the code will not compile.
In other words, putting definitions in headers is a very bad idea. You should stick to declarations only, and templates.
As for templates, compilers know that you may include the same header more than once, they will not generate the same code over and over again.
EDIT: If you mean "keep everything inlined", I think this is a very bad approach. The header files become completely unreadable, and any change in implementation forces any user of your library to recompile everything.
I am developing a Library for Matrix Calculations in C++. For this I wanted to use Templates. After doing a bit of Template Meta Programing, I realized that I would end up in exposing my implementations in the Templatise Matrix Class.Is there any way to obfuscate the template class implementation in the header file when you expose that particular template class ? If yes, then how is it done ?
I will answer from the customer perspective.
When I need to use a library, and integrate it in my code, I expect to see the source code.
It is not because I wish to rip it out from the author... It is not because I am a lawless and irrespective hacker...
It is, simply, because:
code is documentation, and seeing the implementation of a method will help me compensate for the lack of it, or perhaps better understand what it meant (*)
for debugging, the ability to step down into library's code is invaluable
for developing, it is just so much easier if I can compile the code myself, in various flavors (with and without instrumentation, aka gcov, with and without debug symbols, etc...)
I don't ask for the code to be free, I am perfectly fine with the code being licensed, and I'll scrupulously follow the license terms, I just ask for the code to be available.
Frankly, if I have the choice between two libraries, and one does not expose its code, I'll lean toward the other, unless the performance/correctness difference is really important.
(*) In C++, Boost has libraries that I consider fundamentally broken in this regard. The code is riddled with compilers work-around, which makes it very difficult to read. Nevertheless, I use them because they're just awesome.
As templating means that the implementation of the class/function is created compile-time (needs to make a new implementation for each new type) I cannot see how you could hide the code. The only way would be to hide your templates in a precompiled library and only expose interfaces to predefined types. That would lose the template functionality though...
With current standard (and even upcoming C++11), one has to expose all the template definitions where those templates are used. There is no standard way to hide it.
Second part, if you choose to obfuscate it, then equally its usage will become complex. The best way in my opinion is to license/copyright them!
I think all template-based C++ libs are deployed as header files (perhaps also using libraries but the publicly usable templates have to be headers). That's true for STL, boost, etc. It's simply the way templates work -- the compiler has to see the original template.
In addition to all the other reasons cited, there's another problem: C++ names are "decorated" - for example in order to support method overloads, the types of the parameters for the method are encoded in the name of the method.
There is no standard for this encoding, it varies from compiler to compiler and even from one version of a compiler to another version of the same compiler.
As a result, if you have a library containing C++ functions, you can't ensure that the names of the functions can be read by your clients (unless you can guarantee that your clients are using the same version of the compiler that you are).
For standard libraries, this isn't a problem, since the libraries are shipped with the compiler, but for other libraries, you need to be very careful.
No, not really, since a template is not compiled code. It's literally a "template". When a template is instantiated in a .cpp file, the template itself needs to be available to the compiler in order to generate the code for the class method. Therefore you can't "hide" the template code ... it has to be available to the compiler, or else you are going to be unable to compile any modules that attempt to instantiate the template.
One good way to think of templates is like a blank form you might use, say for income-taxes or something of that nature. In order to actually make a valid income-tax form, meaning one filled out with your name, SSN, etc., you need a copy of the "blank" original. So you can't "hide" the form from a person and expect them to fill it out correctly. The same is true for the compiler. When you instantiate a template function or class, a copy of the template needs to be made available for the compiler to fill in the template parameters and actually generate the "real" code behind-the-scenes for you that is then compiled in the code module.
You can put your code in a precompiled header, but I must agree that best way to protect your code is to put license/copyright.
I'm wondering a bit about this templating business.
In C and C++ it is very common to put declarations in header files and definitions in source files, and keep the two completely separate. However, this doesn't even seem to be possible (in any great way) when it comes to templates, and as we all know, templates are a great tool.
Also, Boost is mostly headers, so this is a real issue. Is separating headers and source still a good idea in C++, or should I just not rely heavily on templates?
Instantiating a template is costly at compile time but virtualy free at runtime. Basically, everytime you use a new template type, the compiler has to generate the code for that new type, that's why the code is in a header, so that the compiler have access to the code later.
Putting all your code in a .cpp lets the compiler compile that code only once which greatly speeds up compilation. You could in theory write all your code in headers, it will work fine, but it will take forever to compile very large projects. Also, as soon as you will change one line anywhere, you will have to rebuild everything.
Now you might ask, how comes the STL and BOOST are not so slow? That's where precompiled headers come to the rescue. PCHs let the compiler do the most costly work only once. This works well with code that won't change often like libraries, but its effect is totally nullified for code that changes a lot as you will have to recompile the whole set of precompiled headers everytime. The compiler also use a couple of tricks to avoid recompiling all template code in every compilation unit.
Also note that C++0x will introduce explicit mechanisms to better control template instantiation. You will be able to explicitly instantiate templates and, most importantly, prevent instanciation in some compilation units. However, most of that work is already being done by most compilers without our knowledge.
So, the rule of thumb is, put as much code (and include directives) as possible in your .cpp. If you can't, well, you can't.
My advice would be: don't template just for the heck of it. If you have to template, be careful and be aware that you are in fact choosing between speed of compilation and usability the template will bring.
My personal favourite is this structure:
Header file:
#ifndef HEADER_FILE
#define HEADER_FILE
template < typename T >
class X { void method(); };
#include "header_impl.h"
#endif
Implementation file:
#ifndef HEADER_IMPL_FILE
#define HEADER_IMPL_FILE
#include "header.h"
template < typename T >
void X<T>::method() { }
#endif
I think the really important thing to understand about templates is that, paraphrasing Bjarne Stroustrop, C++ is really like multiple languages rolled into one. The conventions and idioms of templating are different than those of writing "regular" C++, almost like another language.
It is absolutely a good idea to separate header and implementation files in "regular" C++, because the header files tell the compiler what you will supply as an implementation some time later (at link time). It is important because this separation is a very real thing in most common operating systems: link time can happen when the user runs the program. You can compile the implementation into binaries (so's, dll's) and ship the unaltered headers for developers to know how to use your now-opaque implementation.
Now, for templates, you can't do that because the compiler has to fully resolve everything at compile time. That is, when you compile the headers, they have to have the implementation of the template in order for the compiler to be able to make sense of your headers. The compiler essentially "un-templates" your templates when you compile, so there is no option for separating interface and implementation of template C++.
Separation of interface and implementation is good practice, generally, but when you wander into template-land, you are explicitly going to a place where that's just not possible - because that's what a template means: resolve and build the implementation of this interface at compile time, not at runtime.
Here are a couple of techniques I use when writing templated code that move implementations into cpp files:
Move parts of methods that do not depend on the template parameters into separate, non-template helper functions or base classes. The bodies can then go into their own cpp files.
Write declarations of common specializations. The definitions can live in their own cpp files.
Separating header and source is not a C++ idiom, it is more of a C idiom, precisely because C++ favours use of templates and inline functions where possible to reduce the runtime of the program.
I've read a few of the articles about the need / applicability / practicality of keeping headers around in C++ but I can't seem to find anywhere a solid reason why / when the above should or should not be done. I'm aware that boost uses .hpp files to deliver template functions to end users without the need for an associated .cpp file, and this thought is partially sourced off browsing through that code. It seems like this would be a convenient way to deliver single file modules of say a new Wt or Qt widget (still sticking to the one class per .h convention).
However are there any negative technical implementations for giving somebody a single .hpp file with both the header declaration and implementation assuming you have no problem with them having access to the implementation (say in the context of OSS). Does it for instances have any negative implications from the compiler's / linker's perspective?
Any opinions or perspectives on this would be appreciated.
'm aware that boost uses .hpp files to deliver template functions to end users without the need for an associated .cpp file
Wrong verb: it’s not “without the need”, it’s “without the ability”.
If Boost could, they would separate their libraries into headers and implementation files. In fact, they do so where ever possible.
The reason for a clean separation is simple: compilation time for header-only projects increases tremendously because associated header files have to be read, parsed and compiled every time you recompile the tiniest part of your application.
Implementation files only need to be compiled if you happen to recompile that particular object file.
Large C and/or C++ projects take hours to compile. And these use a clean separation into header and object files. If they would only use header files, I’m betting the compilation time would be measured in days instead of hours.
But for many of Boost’s libraries, the fact is that template definitions may not reside in a separate compilation unit than their declarations so this is simply not possible.
The major negative aspect of .hpp-only libraries is that they cannot refer to a precompiled module. All of the code present in the .hpp and hence all of the code in the library must be added to your application. This increases the size of the binary and makes for redundant binaries on such a system that uses the library more than once.
With templates you have no real choice. In theory, export allows you to separate the interface from the implementation, but only one compiler (Comeau) really supports this1, and it's being dropped from C++0x.
In any case, trying to put the implementations of non-template functions into headers leads to one obvious problem: the One Definition Rule remains in effect, so if you define the same function in more than one translation unit, you have a problem. The linker will typically give an error saying the same symbol has been defined more than one.
1Though it's mostly the EDG compiler front-end that really supports it, so other EDG-based compilers, such as Intel's also support export to some degree, though they don't document it, so you can't depend on much with them.