C++ seems to be rather grouchy when declaring templates across multiple files. More specifically, when working with templated classes, the linker expect all method definitions for the class in a single compiler object file. When you take into account headers, other declarations, inheritance, etc., things get really messy.
Are there any general advice or workarounds for organizing or redistributing templated member definitions across multiple files?
Are there any general advice or workarounds for organizing or redistributing templated member definitions across multiple files?
Yes; don't.
The C++ spec permits a compiler to be able to "see" the entire template (declaration and definition) at the point of instantiation, and (due to the complexities of any implementation) most compilers retain this requirement. The upshot is that #inclusion of any template header must also #include any and all source required to instantiate the template.
The easiest way to deal with this is to dump everything into the header, inline where posible, out-of-line where necessary.
If you really regard this as an unacceptable affront, a common option is to split the template into the usual header/implementation pair, and then #include the implementation file at the end of the header.
C++'s "export" feature may or may not provide another workaround. The feature is poorly supported and poorly defined; although it in principle should permit some kind of separate compilation of templates, it doesn't necessarily obviate the demand that the compiler be able to see the entire template body.
Across how many files? If you just want to separate class definitions from implementation then try this article in the C++ faqs. That's about the only way I know of that works at the moment, but some IDEs (Eclipse CDT for example) won't link this method properly and you may get a lot of errors. However, writing your own makefiles or using Visual C++ has always worked for me :-)
When/if your compiler supports C++0x, the extern keyword can be used to separate template declarations from definitions.
See here for a brief explanation.
Also, section 6.3, "The Separation Model," of C++ Templates: The Complete Guide by David Vandevoorde and Nicolai M. Josuttis describes other options.
Related
Bjarne Stroustrup has mentioned the disadvantage of having to define templates in header files multiple times.
Example: https://youtu.be/HddFGPTAmtU
My question is now, if this is now solved by the new concepts feature of C++ in C++20?
I cannot really find anything regarding that and Bjarne did also not say anything about that anymore, afaik.
Concepts itself doesn't eliminate this, but C++ Modules will. C++ Modules are separate from concepts: you can use modules without using concepts. But given that Concepts encourage the creation of generic code, Modules will be a much needed addition.
You'll be able to use Modules side-by-side with #include: you can use one, the other, or both as fits your needs. Modules will speed up the compilation of code significantly, and with modules you'll be able to put templates (and concepts) in a cpp file without even having a header file.
Concepts are about constraining templates and template-related entities. Concepts do not, and has never tried to, address the restriction on having to define templates in header files.
Modules, on the other hand, does try to address that issue. Not by letting you define templates in source files, but by adding a totally new encapsulation layer to the language and hopefully reducing compile times as a result. But while Concepts have already been added to the working draft for C++20, Modules has not been. It's looking like it could make C++20, but it is not yet clear if it will or not. We'll see.
Why the definition of functions is not written in the same "some.h" file together with their declarations? What will be happen if we are not separate "some.h" file from "some.c" file?
So that one knows the minimum he needs to know. (This makes the compilation faster, chances of name collisions lesser, ability to manage code easier etc) As mentioned in this comment by Kerrek SB: If you mix source and code, dividing big projects into modules is difficult.
For example, you can compile a library (Containing definition) and give it to your clients (who care only about the interface) along with the declarations (Headers) and he would be able to use it without needing to know the source of implementation. (This way you can also hide the implementation detail)
Without headers, he doesn't know usage of functions available in library. So though not mandatory, it is recommended to keep the declaration and definitions separate.
Well in fact, separating declarations and definitions is not mandatory, while in practice everything was thought this way to separate usage and implementation, this to promote abstraction, reusability, modularity, etc.
If you don't follow these rules then you will very rapidly face compilation problems with multiple definitions, etc.
We are creating a set of libraries with a public API which is to be used by different third parties. Some of the libraries are pure C so obviously they have a C styled header with functions and struct definitions and the corresponding library. They are ok.
Some of the libraries are written with the usage of a moderately complex C++ (targeting older compilers), so there we have implemented some form of the famous pimpl idiom. This is ok too.
On the other end a significant part of the header files is C++ using heavily templated code. Knowing Why can templates only be implemented in the header file? but also not willing to disclose too much implementation details to eyes who are not supposed to see them we have heavily refactored them to exclude as much internal details as possible and having only the really necessary bits... and there is still a significant amount of code left.
So it puzzles me: Is there a preferred way of distributing header files which largely contain templates? What good practices, best approaches and tips and tricks are there?
Look at your C++ compiler's header files, for an inspiration. The standard C++ library is full of templates, and you will generally find all the template code in the headers.
Having said that, if particular templates are meant to be used with a small number of possible classes (or values) as template parameters, you do have an option of explicitly instantiating templates inside the library itself, leaving just the bare template declarations visible in the header files.
Using a simpler pre-C++11 scenario as an example, a C++ library will typically provide a std::basic_string implementation for only a std::basic_string<char> and std::basic_string<wchar_t>; and leave a bunch of template code inside the library itself, with just a bare std::basic_string template declaration visible in the header files.
I am developing a Library for Matrix Calculations in C++. For this I wanted to use Templates. After doing a bit of Template Meta Programing, I realized that I would end up in exposing my implementations in the Templatise Matrix Class.Is there any way to obfuscate the template class implementation in the header file when you expose that particular template class ? If yes, then how is it done ?
I will answer from the customer perspective.
When I need to use a library, and integrate it in my code, I expect to see the source code.
It is not because I wish to rip it out from the author... It is not because I am a lawless and irrespective hacker...
It is, simply, because:
code is documentation, and seeing the implementation of a method will help me compensate for the lack of it, or perhaps better understand what it meant (*)
for debugging, the ability to step down into library's code is invaluable
for developing, it is just so much easier if I can compile the code myself, in various flavors (with and without instrumentation, aka gcov, with and without debug symbols, etc...)
I don't ask for the code to be free, I am perfectly fine with the code being licensed, and I'll scrupulously follow the license terms, I just ask for the code to be available.
Frankly, if I have the choice between two libraries, and one does not expose its code, I'll lean toward the other, unless the performance/correctness difference is really important.
(*) In C++, Boost has libraries that I consider fundamentally broken in this regard. The code is riddled with compilers work-around, which makes it very difficult to read. Nevertheless, I use them because they're just awesome.
As templating means that the implementation of the class/function is created compile-time (needs to make a new implementation for each new type) I cannot see how you could hide the code. The only way would be to hide your templates in a precompiled library and only expose interfaces to predefined types. That would lose the template functionality though...
With current standard (and even upcoming C++11), one has to expose all the template definitions where those templates are used. There is no standard way to hide it.
Second part, if you choose to obfuscate it, then equally its usage will become complex. The best way in my opinion is to license/copyright them!
I think all template-based C++ libs are deployed as header files (perhaps also using libraries but the publicly usable templates have to be headers). That's true for STL, boost, etc. It's simply the way templates work -- the compiler has to see the original template.
In addition to all the other reasons cited, there's another problem: C++ names are "decorated" - for example in order to support method overloads, the types of the parameters for the method are encoded in the name of the method.
There is no standard for this encoding, it varies from compiler to compiler and even from one version of a compiler to another version of the same compiler.
As a result, if you have a library containing C++ functions, you can't ensure that the names of the functions can be read by your clients (unless you can guarantee that your clients are using the same version of the compiler that you are).
For standard libraries, this isn't a problem, since the libraries are shipped with the compiler, but for other libraries, you need to be very careful.
No, not really, since a template is not compiled code. It's literally a "template". When a template is instantiated in a .cpp file, the template itself needs to be available to the compiler in order to generate the code for the class method. Therefore you can't "hide" the template code ... it has to be available to the compiler, or else you are going to be unable to compile any modules that attempt to instantiate the template.
One good way to think of templates is like a blank form you might use, say for income-taxes or something of that nature. In order to actually make a valid income-tax form, meaning one filled out with your name, SSN, etc., you need a copy of the "blank" original. So you can't "hide" the form from a person and expect them to fill it out correctly. The same is true for the compiler. When you instantiate a template function or class, a copy of the template needs to be made available for the compiler to fill in the template parameters and actually generate the "real" code behind-the-scenes for you that is then compiled in the code module.
You can put your code in a precompiled header, but I must agree that best way to protect your code is to put license/copyright.
I'm wondering a bit about this templating business.
In C and C++ it is very common to put declarations in header files and definitions in source files, and keep the two completely separate. However, this doesn't even seem to be possible (in any great way) when it comes to templates, and as we all know, templates are a great tool.
Also, Boost is mostly headers, so this is a real issue. Is separating headers and source still a good idea in C++, or should I just not rely heavily on templates?
Instantiating a template is costly at compile time but virtualy free at runtime. Basically, everytime you use a new template type, the compiler has to generate the code for that new type, that's why the code is in a header, so that the compiler have access to the code later.
Putting all your code in a .cpp lets the compiler compile that code only once which greatly speeds up compilation. You could in theory write all your code in headers, it will work fine, but it will take forever to compile very large projects. Also, as soon as you will change one line anywhere, you will have to rebuild everything.
Now you might ask, how comes the STL and BOOST are not so slow? That's where precompiled headers come to the rescue. PCHs let the compiler do the most costly work only once. This works well with code that won't change often like libraries, but its effect is totally nullified for code that changes a lot as you will have to recompile the whole set of precompiled headers everytime. The compiler also use a couple of tricks to avoid recompiling all template code in every compilation unit.
Also note that C++0x will introduce explicit mechanisms to better control template instantiation. You will be able to explicitly instantiate templates and, most importantly, prevent instanciation in some compilation units. However, most of that work is already being done by most compilers without our knowledge.
So, the rule of thumb is, put as much code (and include directives) as possible in your .cpp. If you can't, well, you can't.
My advice would be: don't template just for the heck of it. If you have to template, be careful and be aware that you are in fact choosing between speed of compilation and usability the template will bring.
My personal favourite is this structure:
Header file:
#ifndef HEADER_FILE
#define HEADER_FILE
template < typename T >
class X { void method(); };
#include "header_impl.h"
#endif
Implementation file:
#ifndef HEADER_IMPL_FILE
#define HEADER_IMPL_FILE
#include "header.h"
template < typename T >
void X<T>::method() { }
#endif
I think the really important thing to understand about templates is that, paraphrasing Bjarne Stroustrop, C++ is really like multiple languages rolled into one. The conventions and idioms of templating are different than those of writing "regular" C++, almost like another language.
It is absolutely a good idea to separate header and implementation files in "regular" C++, because the header files tell the compiler what you will supply as an implementation some time later (at link time). It is important because this separation is a very real thing in most common operating systems: link time can happen when the user runs the program. You can compile the implementation into binaries (so's, dll's) and ship the unaltered headers for developers to know how to use your now-opaque implementation.
Now, for templates, you can't do that because the compiler has to fully resolve everything at compile time. That is, when you compile the headers, they have to have the implementation of the template in order for the compiler to be able to make sense of your headers. The compiler essentially "un-templates" your templates when you compile, so there is no option for separating interface and implementation of template C++.
Separation of interface and implementation is good practice, generally, but when you wander into template-land, you are explicitly going to a place where that's just not possible - because that's what a template means: resolve and build the implementation of this interface at compile time, not at runtime.
Here are a couple of techniques I use when writing templated code that move implementations into cpp files:
Move parts of methods that do not depend on the template parameters into separate, non-template helper functions or base classes. The bodies can then go into their own cpp files.
Write declarations of common specializations. The definitions can live in their own cpp files.
Separating header and source is not a C++ idiom, it is more of a C idiom, precisely because C++ favours use of templates and inline functions where possible to reduce the runtime of the program.