I'm developing a C++ library that makes heavy uses of templates. For most of the classes, all the code resides in some .hpp file that will be #included directly by client code. There are two things that I'm worrying about:
Is there some design pattern for moving implementation code to the "background"? When I design non-template classes I often separate the "interface" class and the implementation class, and the former will delegate to the latter by doing something like getImpl()->foo(), so that implementation code can be dynamically linked to. I'm not sure how to do this for templates. Probably dynamic linking just doesn't make sense for template, I think?
Are big template headers (> 1000 lines) common? Or is that bad? If it's bad, what can I do with it?
I am aware of this question How to reduce output size of template-heavy C++ code?, but I think we are asking about different things: the OP of that question is trying to reduce the size of the output, while I am trying to reduce the size of my library headers themselves.
UPDATE: For example, if you were to design std::vector, how would you organize its interface & implementation (if needed)?
Don't worry too much about the header size. Templates almost always have all their code in a header because each client must know how instantiate the class in case they are creating a completely new version (with different template parameters).
If you're worried about 1000 lines, take a look at vector. In Visual Studio 2013, it's 3000 lines.
Having said that, you really only need the declaration of classes available everywhere and the definition once, like any normal class. So it is possible to have a template header with no implementation and then create a .cpp with the implementations. This .cpp will have to force the generation of all the templated types that you need everywhere.
In the example below, the author of B.cpp must know all template parameters that will ever be used. You don't need this many files, but I think this should cover every scenario.
B.h:
template<class T>
struct B
{
B();
};
B_impl.h:
#include "B.h"
template<class T>
B<T>::B() {}
B.cpp:
#include "B_impl.h"
template struct B<int>;
template struct B<short>;
main.cpp:
#include "B.h"
B<int> b;
Related
Background:
I currently have a game engine project structured in visual studio so that my 'Engine' is compiled into a dll for another project 'Game' to consume. The idea being I can swap out different game projects and still use the same engine dll code. Within my engine code I'm creating my own framework for which all other engine code will use. This will help to separate my implementation from the rest of my code and make modification easier if need be.
Since all my framework code with be used within the dll itself and not within 'Game' I thought I could implement templates. However, I still receive the 'undefined symbol' error anytime I try and implement templates with the Engine framework.
Issue:
Is there a way to get around the 'undefined symbol' errors for templates from the linker within my dll without having to explicitly define every type my template will consume (ex. class template MyClass<int>, class template MyClass<float>, etc.)? If not, are there any suggestions on different ways I could implement my engine and different game projects to still keep things flexible? Thanks for any input.
P.S. I don't want to have to explicitly define all types a templated class could use since that would get rather large if I wanted to create say my own vector template class (as I would have to define A LOT of different classes).
Template intantiations are generated at compile time, so if you don't want to individually declare all possible instantiations then you must move all template definitions to be included with the header file.
One common way of doing this is for each class to have all of the template definitions in an inline file e.g a .inl or .cxx file, and then to include that inline file at the end of the corresponding header file.
For example, for a class Foo with a single templated function:
Contents of Foo.hpp:
class Foo
{
template <typename T>
void bar();
};
#include "Foo.inl" // Include template definitions
Contents of Foo.inl:
template <typename T>
void Foo::bar()
{
// body
}
That way, whenever Foo::bar<T> is used with a new T then an a new template instantiation will be generated.
Templates are supposed to be inline code. Their code created at code generation step when compiler encounters use of template, with code for that use case. In general attempt to force compiler to create class based on template is a fallacy , performance issue and inconvenience that opposes the purpose of having templates.
In general you need to declare object (variable) of specialized template to force compiler to generate one, if you have limited count of specialized variants< you need to declare them all. Even in that case compilers try avoid of creating new functions, unless they are virtual.
Most template libraries out there fully declare code of methods in headers, and may have no binary object files at all. Just do like mock_blatt said in comment, make declarations of templates in header and implementations of methods and friends in include file used from header.
"I don't want to have to explicitly define all types a templated class could use since that would get rather large"
What exactly do you expect the DLL to contain? Either it contains MyClass<float> or it doesn't. And the compiler would instantiate MyClass<T> for every type it knows, your DLL would be huge.
I'm a noob in C++ meta programming and I understand that I must place implementation of class in header file for convention. But when you want to purchase an package you need to separate the header and the implementation.
Now, Is there a way to do this?
Note: I know about 'export' keyword and read this page!
No, not really. Templates must be visible at compile-time so, if your users are to instantiate templates themselves, they need the definitions.
You'll just have to explicitly instantiate for as many template-parameter-list combinations as you feel you'll need, and hope that your users don't need any more.
Think of templates as something that helps you inside a single project. This inherent limitation means that they are not appropriate as a "I can provide my library for the user to specialise with any type they want" mechanism.
There are several vendors providing general templates for purchase (BTW, I assume you want to sell the template code rather than purchase it) and they simply ship the source. The license agreement states that you can't distribute or modify the code. Done.
If that's not good enough for your needs here are a few things you can do:
Factor code into components with minimal dependency on template arguments. This may yield parts of the code you can put into a readily compiled object because they don't depend on template arguments or you can preinstantiate the code.
Obfuscating the code may be one way to go. In this context it is worth noting that a reason given for the absense of an "Obfuscated C++ Contest" by Steve Clamage was: "That would be like shooting fish in a barrel."
Create a service where customers upload the class declarations needed for template instantiation and get back the compiled instantiation. Of course, if you don't trust the customers they have no reason to assume that you might be trustworthy.
Wait for the next revision of the standard which hopefully has some sort of module support helping with this issue. I wouldn't bet too high on this, however.
Personally, I wouldn't bother: from feedback I get back on my code it seems that despite documentation and explanations nobody is prepared to touch it. I think it is rather readable..,
templatelib.h
template<typename T> void foo();
templatelib.cpp
#include <iostream>
#include "templatelib.h"
template<typename T> void foo() {
std::cout << typeid(T).name() << '\n';
}
template void foo<int>(); // explicitly instantiate template for this type
main.cpp
#include "templatelib.h"
int main() {
foo<int>(); // Okay
foo<double>(); // error
}
Not really. You can obfuscate your code, you can separate the implementation in a impl file and include it in the header, but it must be visible to the compiler.
However, if there's a way you can restrict how many actual specializations are used, you can move specialized implementations in a cpp file, keep the declaration in the header, and it would still work:
header.h
template<typename X>
void foo(X x);
template<>
void foo<int>(int x);
impl1.cpp
#include "header.h"
template<>
void foo<int>(int x)
{
}
impl2.cpp
#include "header.h"
foo(1); //works
Also, I don't think you should worry about this. Because they are generic, I don't think templates can have that much of an important logic inside of them. Take a look at the standard ones - vector, list - sure, they would take some time to write from scratch, but there's no real commercial secret in there.
This question already has answers here:
Why can templates only be implemented in the header file?
(17 answers)
Closed 9 years ago.
Normally when I create a class, I create a header and a source for that class. I have heard that with a template class, you have to put the function implementation in the header. I tried doing it both ways, and got compilation errors the first way. The second way worked fine. However, I like to organize my code into headers and source files, so is it possible to put the function implementations into a source file? (Maybe it requires special compilation flags or syntax?) Or should I just keep em in the header?
Thanks!
Generally, all template code must be in a header file since the compiler needs to know the complete type at the point of instantiation.
As Aaron says below it is possible to put the implementation details in a .cpp-file in the specific case where you know on before hand all possible types the template will be instantiated with and explicitly instantiate it with those types. You'll then get a linker error if the template gets instantiated with another type somewhere in your code.
A quite common general solution to at least visually separate interface from implementation is to put all implementation in a .inc (or .tcc or .ipp)-file and include it at the end of the header file.
Note that the syntax for putting template class members outside the class-definition (whether you use the specific solution or the general) is slightly cumbersome. You'll need to write:
// in test.h
template <class A>
class Test
{
public:
void testFunction();
};
#include "test.inc"
// in test.inc
template <class A>
void Test<A>::testFunction()
{
// do something
}
(Edited: This is slightly more robust version which allows the implementation to be compiled into a separate .o file. The template should be explicitly instantiated at the end of the implementation cpp file. Perhaps this was only an issue for g++.)
You don't need to put the implementations in the header if you know which templates will be instantiated and can list them in the header implementation file. For example, if you know that you will only use int and std::string, then you can put this in the header file:
// test.h
template <class A>
class Test
{
public:
void f();
};
and put the implementation of f() into a normal test.cpp file:
// test.cpp
#include "test.h"
template <class A> void Test<A>::f() {
// implementation
}
template class Test<int>;
template class Test<string>;
Those last two lines explicitly instantiate the template classes. It is better to put this at the end of the implementation file, after it has seen the implementations of the member functions. Then you can compile it to a .o file g++ -c test.cpp. This test.o file will contain complete implementations of both Test<int> and Test<string> and can be linked without any difficulty in the rest of your application.
It works, but is it a good idea? It depends on context. In many cases, this works very well. If you are writing a template for 'internal' use in your project, then you know which templates will be instantiated and which will not. But if instead you're making something available to the public which must be very flexible, then you will need to include the implementations in the header file.
A tip: Even if it's for public consumption, take a look at the methods and see if there are any methods whose parameters and return type are independent of the template parameters. If so, you could place them into a Base class as (pure) virtual functions. This Base class doesn't use any templates and hence you may be able to use this Base* in much of your application (template <class A> class Test : public Base { ..., allowing you to limit the reach of the templated code throughout your application. I found this useful recently when much of the underlying behaviour and construction of a class depended on the template parameters, but the interface to the already-constructed object did not depend on the template parameters.
To answer the original question: no, the definitions of [member] function templates don't have to go into the header. However, the compiler needs to see the definition to instantiate the template. For templates instantiated with many different types you want the templates to be implicitly instantiated when they are used. This is e.g. the case for class templates like std::vector<...> and for function templates like std::copy(...). In this case it is almost certainly impractical to separate the template definition from its declaration although I personally put the definitions into a separate file included at the bottom of the header file.
For templates instantiated with only a few types like the stream classes or std::basic_string<...> it is often better to define the function templates in a separate header-like file which is only included in implementation files which explicitly instantiates them. This way the effort of instantiating the template is only spent once rather than in every translation unit using it. Especially for the stream classes this makes a huge difference (primarily for compile and link time but on some systems also for executable size). ... and I'm pretty sure hardly anybody has gone to the trouble of using the stream classes with different character types than char and wchar_t (hint: it is non-trivial to arrange for the various facets to be implemented and present in the std::locale). The technique of explicitly instantiating templates also works well if there is only a limited set of types with which the template is intended to work.
While it is technically the case that you can separate your implementation from your interface the syntax for templates gets so annoying to type repeatedly I would strongly recommend that you just hold your nose and put the implementation in your class until you get over the smell.
template <class X>
class klass_t {
public:
void f();
void g();
void h();
};
template <class X>
void klass_t<X>::f() { ... }
template <class X>
void klass_t<X>::g() { ... }
template <class X>
void klasS_t<X>::h() { ... }
Would have been:
template <class X>
class klass_t {
public:
void f() { ... }
void g() { ... }
void h() { ... }
};
Now imagine you want to add another template parameter (as in the following). Now you have to change the template argument lists in n places rather than just one.
template <class X, class Y>
class klass_t {
public:
void f();
void g();
void h();
};
Unless you have a good reason not to, it is a lot easier on your fingers to just put everything in the class.
Template implementations need to be known at compile-time. That means the implementations must be visible to the compiler. There's no way to go around this.
If you want to keep implementation details a secret, there's no way to do this. You can of course obfuscate your code, but that's not much of an impediment.
If your only concern is code organization, you can create a separate include file with the implementations and include it at the end of your main header (much as Andreas' suggestion).
I am in the process of designing a C++ static library.
I want to make the classes generic/configuarable so that they can support a number of data types(and I don't want to write any data type specific code in my library).
So I have templatized the classes.
But since the C++ "export" template feature is not supported by the compiler I am currently using, I am forced to provide the implementation of the classes in the header file.
I dont want to expose the implementation details of my Classes to the client code which is going to use my library.
Can you please provide me with some design alternatives to the above problem??
Prior to templates, type-agnostic C++ code had to be written using runtime polymorphism. But with templates as well, you can combine the two techniques.
For example, suppose you wanted to store values of any type, for later retrieval. Without templates, you'd have to do this:
struct PrintableThing
{
// declare abstract operations needed on the type
virtual void print(std::ostream &os) = 0;
// polymorphic base class needs virtual destructor
virtual ~PrintableThing() {}
};
class PrintableContainer
{
PrintableThing *printableThing;
public:
// various other secret stuff
void store(PrintableThing *p);
};
The user of this library would have to write their own derived version of PrintableThing by hand to wrap around their own data and implement the print function on it.
But you can wrap a template-based layer around such a system:
template <T>
struct PrintableType : PrintableThing
{
T instance;
virtual void print(std::ostream &os)
{ os << instance; }
PrintableType(const T &i)
: instance(i) {}
};
And also add a method in the header of the library, in the declaration of the PrintableContainer class:
template <class T>
void store(const T &p)
{
store(new PrintableType(p));
}
This acts as the bridge between templates and runtime polymorphism, compile-time binding to the << operator to implement print, and to the copy-constructor also (and of course also forwarding to the nested instance's destructor).
In this way, you can write a library entirely based on runtime polymorphism, with the implementation capable of being hidden away in the source of the library, but with a little bit of template "sugar" added to make it convenient to use.
Whether this is worth the trouble will depend on your needs. It has a purely technical benefit in that runtime polymorphism is sometimes exactly what you need, in itself. On the downside, you will undoubtedly reduce the compiler's ability to inline effectively. On the upside, your compile times and binary code bloat may go down.
Examples are std::tr1::function and boost::any, which have a very clean, modern C++ template-based front end but work behind the scenes as runtime polymorphic containers.
I've got some news for you, buddy. Even with export, you'd still have to release all of your template code -- export just makes it that you don't have to put the definitions in a header file. You're totally stuck. The only technique you can use is split off some functions that are non-templates and put them into a different class. But that's ugly, and usually involves void* and placement new and delete. That's just the nature of the beast.
You can try to obfuscate your code - but you have little choice in C++03 asides from including template code in header files.
Vandevoorde does describe another technique in his book: Explicit instantiation - but that entails having to explicitly instantiate all possible useful combinations.
But for the most comprehensive review of this topic read chapter 6 from C++ Templates: The Complete Guide.
Edit (in response to your comment): You have two options for writing generic code without using templates:
1) Preprocessor - still requires header files
2) using void* - yuk - incredibly unsafe
So no, i do not recommend not using templates for solving problems that templates were specifically designed (albeit somewhat flawed) for.
One problem with templates is that they require compiled code. You never know how the end-user will specialize/instantiate your templates, so your dll-file would have to contain all possible template specializations in compiled form. This means that to export pair<X,Y> template you would have to force the compilication of pair<int,float>, pair<int,string>, pair<string,HWND> and so on... to infinity..
I guess more practical solution for you would be to un-template private/hidden code. You can create special internal functions that would only be compiled for single template specialization. In the following example internal_foo-function is never called from MyClass where A is not int.
template<class A>
class MyClass
{
int a;
float b;
A c;
int foo(string param1);
{
((MyClass<int>*)this)->internal_foo(param1);
}
int internal_foo(string param1); // only called on MyClass<int> instances
};
template<>
__declspec(dllexport) int MyClass<int>::internal_foo(string param1)
{
... secret code ...
}
This of course is a hack. When using it you should be extra careful not to use member variable "c", because it's not always integer (even though internal_foo thinks that it is). And you can't even guard yourself with assertions. C++ allows you to shoot yourself in the foot, and gives you no indications about it until it's too late.
PS. I haven't tested this code so it might require some fine tuning. Not sure for example if __declspec(dllimport) is needed in order for compiler to find internal_foo function from dll-file...
With templates you cannot avoid shipping the code (unless your code only works with a fixed set of types, in which case you can explicitly instantiate). Where I work we have a library that must work on POD types (CORBA/DDS/HLA data definitions), so at the end we ship templates.
The templates delegate most of the code to non-templated code that is shipped in binary form. In some cases, work must be done directly in the types that were passed to the template, and cannot thus be delegated to non-templated code, so it is not a perfect solution, but it hides enough part of the code to make our CEO happy (the people in charge of the project would gladly provide all the code in templates).
As Neil points in a comment to the question, in the vast majority of cases there is nothing magical in the code that could not be rewritten by others.
I want to remove, if possible, the includes of both <vector> and <string> from my class header file. Both string and vector are return types of functions declared in the header file.
I was hoping I could do something like:
namespace std {
template <class T>
class vector;
}
And, declare the vector in the header and include it in the source file.
Is there a reference covering situations where you must include in the header, and situations where you can pull the includes into the source file?
You cannot safely forward declare STL templates, at least if you want to do it portably and safely. The standard is clear about the minimum requirements for each of the STL element, but leaves room for implemtation extensions that might add extra template parameters as long as those have default values. That is: the standard states that std::vector is a template that takes at least 2 parameters (type and allocator) but can have any number of extra arguments in a standard compliant implementation.
What is the point of not including string and vector headers? Surely whoever is going to use your class must have already included it since it is on your interface.
When you ask about a reference to decide when to include and when to forward declare, my advice would be: include everything that is part of your interface, forward declare internal details.
There are more issues here that plain compilation performance. If you push the include of a type that is in your public (or protected) interface outside of the header you will be creating dependencies on the order of includes. Users must know that they must include string before including your header, so you are giving them one more thing to worry about.
What things should be included in the implementation file: implementation details, loggers, elements that don't affect the interface (the database connectors, file headers), internal implementation details (i.e. using STL algorithms for your implementation does not affect your interface, functors that are created for a simple purpose, utilities...)
With a very few exceptions, you are not allowed to add things to the std:; namespace. For classes like vector and string, you therefore have no option but to #include the relevant Standard header files.
Also, notice that string is not a class, but a typedef for basic_string<char>.
This won't help for vector or string, but it might be worth mentioning that there is a forward reference header for iostream, called iosfwd.
Standard containers often have additional default template parameters (allocators, etc.) so this will not work. For example, here's a snippet from GNU implementation:
template<typename _Tp, typename _Alloc = std::allocator<_Tp> >
class vector : protected _Vector_base<_Tp, _Alloc>
{ ... };
This was something I was trying to do earlier too, but this is not possible due to templates.
Please see my question: Forward Declaration of a Base Class
As such headers don't change during your development it's not worth optimizing that anyway...
There is no simple obvious way to do it (as others have explained it very well).
However these headers should be seen as being part of the language (really!), so you can let them in your own headers without any problem, nobody will ever complain.
If your concern is compilation speed, I encourage you to use pre-compiled header instead and put these std headers in it (among other things). It will significantly increase your compilation speed.
Sorry the for the "real winner is the one who avoid the fight" kind of answer.
Just include the header in any file where you reference an STL collection.
As others have mentioned, there's not a way to reliably forward declare the STL classes, and even if you find one for your particular implementation, it will probably break if you use a different STL implementation.
If the compilation units don't instantiate the classes, it won't make your object files any bigger.
If string and vector are used only in signatures of non-public members of you class, you could use the PImpl idiom:
// MyClass.h
class MyClassImpl;
class MyClass{
public:
MyClass();
void MyMethod();
private:
MyClassImpl* m_impl;
};
// MyClassImpl.h
#include <vector>
#include <string>
#include <MyClass.h>
class MyClassImpl{
public:
MyClassImpl();
void MyMethod();
protected:
std::vector<std::string> StdMethod();
};
// MyClass.cpp
#include <MyClass.h>
#include <MyClassImpl.h>
void MyClass::MyMethod(){
m_impl->MyMethod();
}
You are always including vector and string in the header file, but only in the implementation part of your class; files including only MyClass.h will not be pulling in string and vector.
WARNING
Expect that doing this will cause uproar.
The language allows you to derive your own classes:
// MyKludges.h
#include <vector>
#include <string>
class KludgeIntVector : public std::vector<int> {
// ...
};
class KludgeDoubleVector : public std::vector<double> {
// ...
};
class KludgeString : public std::string {
// ...
};
Change your functions to return KludgeString and KludgeIntVector. Since these are no longer templates, you can forward declare them in your header files, and include MyKludges.h in your implementation files.
Strictly speaking, derived classes do not inherit base class constructors, destructors, assignment operators, and friends. You will need to provide (trivial) implementations of any that you're using.
// LotsOfFunctions.h
// Look, no includes! All forward declared!
class KludgeString;
// 10,000 functions that use neither strings nor vectors
// ...
void someFunction(KludgeString &);
// ...
// Another 10,000 functions that use neither strings nor vectors
// someFunction.cpp
// Implement someFunction in its own compilation unit
// <string> and <vector> arrive on the next line
#include "MyKludges.h"
#include "LotsOfFunctions.h"
void someFunction(KludgeString &k) { k.clear(); }
Maybe you would better use the pimpl idiom: it appears to me that you don't want to expose the implementation of your class to client code. If the vector and string objects are aggregated by value, the compiler needs to see their full declarations.
With the exception of adding overloads to std::swap (the only exception I can think of right now), you are generally not allowed to add anything to the std namespace. Even if it were allowed, the actual declaration for std::vector is a lot more complicated than the code in the OP. See Nikolai N Fetissov's answer for an example.
All that aside, you have the additional problem of what your class users are going to do with functions that return a std::vector or std::string. The C++ Standard section 3.10 says that functions returning such objects are returning rvalues, and rvalues must be of a complete type. In English, if your users want to do anything with those functions, they'll have to #include <vector> or <string> anyway. I think it would be easier to #include the headers for them in your .h file and be done with it.
I assume your objective here is to speed up compile times? Otherwise I'm not sure why you would want to remove them.
Another approach (not pretty but practical) is to use macro guards around the include itself.
e.g.
#ifndef STDLIB_STRING
#include <string>
#define STDLIB_STRING
#endif
Although this looks messy, on large codebases it does indeed increase the compile times. What we did is create a Visual Studio macro that will automatically generate the guards. We bind the macro to a key for easy coding. Then it just becomes a company coding standard (or habit).
We also do it for our own includes as well.
e.g.
#ifndef UTILITY_DATE_TIME_H
#include "Utility/DateTime.h"
#endif
Since we have Visual Studio helpers to auto-generate the guards when we create our own header files, we don't need the #define. The macro knows it's a internal include because we always use the
#include ""
format for our own includes and
#include <>
for external includes.
I know it doesn't look pretty but it did speed up our compile times on a largish codebase by over 1/2 hour (from memory).