We are creating a set of libraries with a public API which is to be used by different third parties. Some of the libraries are pure C so obviously they have a C styled header with functions and struct definitions and the corresponding library. They are ok.
Some of the libraries are written with the usage of a moderately complex C++ (targeting older compilers), so there we have implemented some form of the famous pimpl idiom. This is ok too.
On the other end a significant part of the header files is C++ using heavily templated code. Knowing Why can templates only be implemented in the header file? but also not willing to disclose too much implementation details to eyes who are not supposed to see them we have heavily refactored them to exclude as much internal details as possible and having only the really necessary bits... and there is still a significant amount of code left.
So it puzzles me: Is there a preferred way of distributing header files which largely contain templates? What good practices, best approaches and tips and tricks are there?
Look at your C++ compiler's header files, for an inspiration. The standard C++ library is full of templates, and you will generally find all the template code in the headers.
Having said that, if particular templates are meant to be used with a small number of possible classes (or values) as template parameters, you do have an option of explicitly instantiating templates inside the library itself, leaving just the bare template declarations visible in the header files.
Using a simpler pre-C++11 scenario as an example, a C++ library will typically provide a std::basic_string implementation for only a std::basic_string<char> and std::basic_string<wchar_t>; and leave a bunch of template code inside the library itself, with just a bare std::basic_string template declaration visible in the header files.
Related
I just noted this item in the Google C++ Coding Style Guide - and I don't quite get it.
If I put an inline method or function in a file other than the header included by other files, it will not be a method of the class; and it will only be usable to bits of code which include it. So why even have such -inl.h files at all?
Also, why do we even want to inline long functions anyways? (i.e. other than in the case of templates, when we have to put the code in the header files for instantiation)
I just noted this item in the Google C++ Coding Style Guide - and I don't quite get it.
Do take that guide with a pinch of salt. Many of the guidelines are intended to help interact with Google's legacy codebase, and aren't particularly good advice for general C++ development.
So why even have such -inl.h files at all?
There's no particularly good reason; I don't do that myself. Some people like them because it minimises the amount of stuff in the main header file, which users of the header generally want to read, and separates out the implementation details, which they generally don't care about.
Also, why do we even want to inline long functions anyways?
Sometimes, we must: template definitions must be available in any translation unit that instantiates the template, so they (usually) need to be in headers.
Sometimes, we want to: by implementing a function inline in a header, we don't have to worry about building and linking a separate translation unit for it. This can make it more convenient to distribute a library; possibly at the cost of longer build times.
This is often done for long function templates. The regular header my_functions.h only contains the declarations, and the implementation file my_functions-inl.h contain the implementations. The reason is that function templates cannot be put in .cpp files. Note that the X.h file includes the X-inl.h file, not the other way around.
Other libraries have different naming conventions: e.g. some of the Boost libraries use .hpp for template headers and .ipp for template implementation files.
According to latest Google coding style, it is no longer allowed
https://google.github.io/styleguide/cppguide.html#Variable_Names
Prefer placing the definitions for template and inline functions in the same file as their declarations. The definitions of these constructs must be included into every .cc file that uses them, or the program may fail to link in some build configurations. If declarations and definitions are in different files, including the former should transitively include the latter. Do not move these definitions to separately included header files (-inl.h); this practice was common in the past, but is no longer allowed.
Obviously template libraries need to be header only, but for non-templates, when should you make things header-only?
If you think your non-template library could be header-only, consider dividing it into two files anyway, then providing a third file that includes both the .h and the .cpp (with an include guard).
Then anyone who uses your library in a lot of different TUs, and suspects that this might be costing a lot of compile time, can easily make the change to test it.
Once you know users have the option which way to use the library, the answer probably becomes "offer that option whenever you possibly can". So pretty much any time that including it from multiple TUs wouldn't violate the ODR. For instance, if your non-static free functions refer to static globals, then you're out of luck, since the different definitions of that function in different TUs would refer to different objects by the same name, which is an ODR-violation.
You could follow Boost.Asio lead.
They simply provide the two versions of the libraries: header-only and header + library.
They do it with a single macro to be defined (or not) before including their headers. I think the default (if not defined) is to use the header-only version.
See Optional Separate Compilation.
Note how they neatly provide a single source file to be compiled that define everything or the option to link against a dynamically loaded library.
Template libraries need not to be header-only: implementations might well contain some pieces independent of template parameters, and for some reasons (e.g. less code size) separated into a special binary.
I cannot imagine a case where a non-template library really must be header-only. However sometimes it might be reasonable performance-wise to allow inlining of all the code. An example can be a library of wrappers around platform-specific interfaces, e.g. for things like synchronization primitives, thread-local storage, platform- and compiler-specific implementation of atomic operations etc.
Without templates, you'd have actual definitions in the headers. That means that if two files include your header, you'd get multiple definitions and the code will not compile.
In other words, putting definitions in headers is a very bad idea. You should stick to declarations only, and templates.
As for templates, compilers know that you may include the same header more than once, they will not generate the same code over and over again.
EDIT: If you mean "keep everything inlined", I think this is a very bad approach. The header files become completely unreadable, and any change in implementation forces any user of your library to recompile everything.
I am developing a Library for Matrix Calculations in C++. For this I wanted to use Templates. After doing a bit of Template Meta Programing, I realized that I would end up in exposing my implementations in the Templatise Matrix Class.Is there any way to obfuscate the template class implementation in the header file when you expose that particular template class ? If yes, then how is it done ?
I will answer from the customer perspective.
When I need to use a library, and integrate it in my code, I expect to see the source code.
It is not because I wish to rip it out from the author... It is not because I am a lawless and irrespective hacker...
It is, simply, because:
code is documentation, and seeing the implementation of a method will help me compensate for the lack of it, or perhaps better understand what it meant (*)
for debugging, the ability to step down into library's code is invaluable
for developing, it is just so much easier if I can compile the code myself, in various flavors (with and without instrumentation, aka gcov, with and without debug symbols, etc...)
I don't ask for the code to be free, I am perfectly fine with the code being licensed, and I'll scrupulously follow the license terms, I just ask for the code to be available.
Frankly, if I have the choice between two libraries, and one does not expose its code, I'll lean toward the other, unless the performance/correctness difference is really important.
(*) In C++, Boost has libraries that I consider fundamentally broken in this regard. The code is riddled with compilers work-around, which makes it very difficult to read. Nevertheless, I use them because they're just awesome.
As templating means that the implementation of the class/function is created compile-time (needs to make a new implementation for each new type) I cannot see how you could hide the code. The only way would be to hide your templates in a precompiled library and only expose interfaces to predefined types. That would lose the template functionality though...
With current standard (and even upcoming C++11), one has to expose all the template definitions where those templates are used. There is no standard way to hide it.
Second part, if you choose to obfuscate it, then equally its usage will become complex. The best way in my opinion is to license/copyright them!
I think all template-based C++ libs are deployed as header files (perhaps also using libraries but the publicly usable templates have to be headers). That's true for STL, boost, etc. It's simply the way templates work -- the compiler has to see the original template.
In addition to all the other reasons cited, there's another problem: C++ names are "decorated" - for example in order to support method overloads, the types of the parameters for the method are encoded in the name of the method.
There is no standard for this encoding, it varies from compiler to compiler and even from one version of a compiler to another version of the same compiler.
As a result, if you have a library containing C++ functions, you can't ensure that the names of the functions can be read by your clients (unless you can guarantee that your clients are using the same version of the compiler that you are).
For standard libraries, this isn't a problem, since the libraries are shipped with the compiler, but for other libraries, you need to be very careful.
No, not really, since a template is not compiled code. It's literally a "template". When a template is instantiated in a .cpp file, the template itself needs to be available to the compiler in order to generate the code for the class method. Therefore you can't "hide" the template code ... it has to be available to the compiler, or else you are going to be unable to compile any modules that attempt to instantiate the template.
One good way to think of templates is like a blank form you might use, say for income-taxes or something of that nature. In order to actually make a valid income-tax form, meaning one filled out with your name, SSN, etc., you need a copy of the "blank" original. So you can't "hide" the form from a person and expect them to fill it out correctly. The same is true for the compiler. When you instantiate a template function or class, a copy of the template needs to be made available for the compiler to fill in the template parameters and actually generate the "real" code behind-the-scenes for you that is then compiled in the code module.
You can put your code in a precompiled header, but I must agree that best way to protect your code is to put license/copyright.
I'm wondering a bit about this templating business.
In C and C++ it is very common to put declarations in header files and definitions in source files, and keep the two completely separate. However, this doesn't even seem to be possible (in any great way) when it comes to templates, and as we all know, templates are a great tool.
Also, Boost is mostly headers, so this is a real issue. Is separating headers and source still a good idea in C++, or should I just not rely heavily on templates?
Instantiating a template is costly at compile time but virtualy free at runtime. Basically, everytime you use a new template type, the compiler has to generate the code for that new type, that's why the code is in a header, so that the compiler have access to the code later.
Putting all your code in a .cpp lets the compiler compile that code only once which greatly speeds up compilation. You could in theory write all your code in headers, it will work fine, but it will take forever to compile very large projects. Also, as soon as you will change one line anywhere, you will have to rebuild everything.
Now you might ask, how comes the STL and BOOST are not so slow? That's where precompiled headers come to the rescue. PCHs let the compiler do the most costly work only once. This works well with code that won't change often like libraries, but its effect is totally nullified for code that changes a lot as you will have to recompile the whole set of precompiled headers everytime. The compiler also use a couple of tricks to avoid recompiling all template code in every compilation unit.
Also note that C++0x will introduce explicit mechanisms to better control template instantiation. You will be able to explicitly instantiate templates and, most importantly, prevent instanciation in some compilation units. However, most of that work is already being done by most compilers without our knowledge.
So, the rule of thumb is, put as much code (and include directives) as possible in your .cpp. If you can't, well, you can't.
My advice would be: don't template just for the heck of it. If you have to template, be careful and be aware that you are in fact choosing between speed of compilation and usability the template will bring.
My personal favourite is this structure:
Header file:
#ifndef HEADER_FILE
#define HEADER_FILE
template < typename T >
class X { void method(); };
#include "header_impl.h"
#endif
Implementation file:
#ifndef HEADER_IMPL_FILE
#define HEADER_IMPL_FILE
#include "header.h"
template < typename T >
void X<T>::method() { }
#endif
I think the really important thing to understand about templates is that, paraphrasing Bjarne Stroustrop, C++ is really like multiple languages rolled into one. The conventions and idioms of templating are different than those of writing "regular" C++, almost like another language.
It is absolutely a good idea to separate header and implementation files in "regular" C++, because the header files tell the compiler what you will supply as an implementation some time later (at link time). It is important because this separation is a very real thing in most common operating systems: link time can happen when the user runs the program. You can compile the implementation into binaries (so's, dll's) and ship the unaltered headers for developers to know how to use your now-opaque implementation.
Now, for templates, you can't do that because the compiler has to fully resolve everything at compile time. That is, when you compile the headers, they have to have the implementation of the template in order for the compiler to be able to make sense of your headers. The compiler essentially "un-templates" your templates when you compile, so there is no option for separating interface and implementation of template C++.
Separation of interface and implementation is good practice, generally, but when you wander into template-land, you are explicitly going to a place where that's just not possible - because that's what a template means: resolve and build the implementation of this interface at compile time, not at runtime.
Here are a couple of techniques I use when writing templated code that move implementations into cpp files:
Move parts of methods that do not depend on the template parameters into separate, non-template helper functions or base classes. The bodies can then go into their own cpp files.
Write declarations of common specializations. The definitions can live in their own cpp files.
Separating header and source is not a C++ idiom, it is more of a C idiom, precisely because C++ favours use of templates and inline functions where possible to reduce the runtime of the program.
C++ seems to be rather grouchy when declaring templates across multiple files. More specifically, when working with templated classes, the linker expect all method definitions for the class in a single compiler object file. When you take into account headers, other declarations, inheritance, etc., things get really messy.
Are there any general advice or workarounds for organizing or redistributing templated member definitions across multiple files?
Are there any general advice or workarounds for organizing or redistributing templated member definitions across multiple files?
Yes; don't.
The C++ spec permits a compiler to be able to "see" the entire template (declaration and definition) at the point of instantiation, and (due to the complexities of any implementation) most compilers retain this requirement. The upshot is that #inclusion of any template header must also #include any and all source required to instantiate the template.
The easiest way to deal with this is to dump everything into the header, inline where posible, out-of-line where necessary.
If you really regard this as an unacceptable affront, a common option is to split the template into the usual header/implementation pair, and then #include the implementation file at the end of the header.
C++'s "export" feature may or may not provide another workaround. The feature is poorly supported and poorly defined; although it in principle should permit some kind of separate compilation of templates, it doesn't necessarily obviate the demand that the compiler be able to see the entire template body.
Across how many files? If you just want to separate class definitions from implementation then try this article in the C++ faqs. That's about the only way I know of that works at the moment, but some IDEs (Eclipse CDT for example) won't link this method properly and you may get a lot of errors. However, writing your own makefiles or using Visual C++ has always worked for me :-)
When/if your compiler supports C++0x, the extern keyword can be used to separate template declarations from definitions.
See here for a brief explanation.
Also, section 6.3, "The Separation Model," of C++ Templates: The Complete Guide by David Vandevoorde and Nicolai M. Josuttis describes other options.