I was (and have been for a long time) under the impression that you had to fully define all template functions in your .h files to avoid multiple definition errors that occur due to the template compilation process (non C++11).
I was reading a co-worker's code, and he had a non-template class that had a template function declared in it, and he separated the function declaration from the function definition (declared in H, defined in CPP). It compiles and works fine to my surprise.
Is there a difference between how a template function in a non template class is compiled, and how a function in a template class is compiled? Can someone explain what that difference is or where I might be confused?
The interesting bit is how and when the template gets instantiated. If the instantiations can be found at link time, the template definition doesn't need to be visible in the header file.
Sometimes, explicit instantiations are cause like this:
header :
struct X {
// function template _declaration_
template <typename T> void test(const T&);
};
cpp:
#include "X.h"
// function template _definition_:
template <typename T>
void X::test(const T&)
{
}
// explicit function template _instantiation(s)_:
template X::test<int>(const int&);
template X::test<std::string>(const std::string&);
Using this sample, linking will succeed unless uninstantiated definitions of the template are used in other translation units
There is no difference between function templates defined in namespace or in class scope. It also doesn't matter whether is inside class template or not. What matter is that at some point in the project any used function template (whether member or non-member) is instantiated. Let's go over the different situations:
Unused function templates don't need to be instantiated and thus their implementation doesn't need to be visible to compiler at any point in time. This sounds boring but is important e.g. when using SFINAE approaches where class or function templates are declared but not defined.
Any function template which is defined where it is used will be instantiated by the compiler in a form which allows multiple definitions across different translation units: only one of the instantiations is retained. It is important that all the different definitions are merged because you could detect differences if you took the address of a function template or used a state variable inside the function template: there shall be only one of these for each instantiation.
The most interesting setup is where the definition of the function template is not seen when the function template is used: in this case the compiler cannot instantiate it. When the compiler sees a definition of the function template in a different translation unit it wouldn't know which template arguments to instantiate! A Catch 22? Well, you can always explicitly instantiate a template once. Having multiple explicit instantiations would create multiply defined symbols.
These are roughly the important options. There are often good reasons that you don't want to have the definition of a function template in the header. For example, you don't necessarily want to drag in dependencies you wouldn't have otherwise. Putting the definition of the function template somewhere else and explicitly instantiating it is a good thing. Also, you might want to reduce the compile time e.g. by avoiding to instantiate essentially the entire I/O stream and locale library in every translation unit using the stream. In C++ 2011 extern templates were introduced which allow the declaration that a particular template (either function or class template) is defined externally once for the entire program and there isn't any need to instantiate it in every header using particularly common template arguments.
For a longer version of what I just said, including examples have a look at a blog post
I wrote last weekend on this topic.
Related
I'm reading a book about how templates work, and I'm having difficulty understanding this explanation of templates.
It says
When the compiler sees the definition of a template, it does not generate code. It generates code only when we instantiate a specific instance of the template. The fact that code is generated only when we use a template (and not when we define it) affects how we organize our source code and when errors are detected...To generate an instantiation, the compiler needs to have the code that defines a function template or class template member function. As a result, unlike non-template code, headers for templates typically include definitions as well as declarations.
What exactly does it mean by "generate code"? I don't understand what is different when you compile function templates or class templates compared to regular functions or classes.
The compiler generates the code for the specific types given in the template class instantiation.
If you have for instance a template class declaration as
template<typename T>
class Foo
{
public:
T& bar()
{
return subject;
}
private:
T subject;
};
as soon you have for example the following instantiations
Foo<int> fooInt;
Foo<double> fooDouble;
these will effectively generate the same linkable code as you would have defined classes like
class FooInt
{
public:
int& bar()
{
return subject;
}
private:
int subject;
}
and
class FooDouble
{
public:
double& bar()
{
return subject;
}
private:
double subject;
}
and instantiate the variables like
FooInt fooInt;
FooDouble fooDouble;
Regarding the point that template definitions (don't confuse with declarations regardless of templates) need to be seen with the header (included) files, it's pretty clear why:
The compiler can't generate this code without seeing the definition. It can refer to a matching instantiation that appeared first at linking stage though.
What does a non-template member function have that allows for it to
be defined outside of the header that a template function doesn't
have?
The declaration of a non-template class/member/function gives a predefined entry point for the linker. The definition can be drawn from a single implementation seen in a compiled object file (== .cpp == compilation unit).
In contrast the declaration of a templated class/member/function might be instantiated from arbitrary compilation units given the same or varying template parameters. The definition for these template parameters need's to be seen at least once. It can be either generic or specialized.
Note that you can specialize template implementations for particular types anyway (included with the header or at a specific compilation unit).
If you would provide a specialization for your template class in one of your compilation units, and don't use your template class with types other than specialized, that also should suffice for linking it all together.
I hope this sample helps clarifying what's the difference and efforts done from the compiler.
A template is a pattern for creating code. When the compiler sees the definition of a template it makes notes about that pattern. When it sees a use of that template it digs out its notes, figures out how to apply the pattern at the point where it's being used, and generates code according to the pattern.
What is the compiler suppose to do when it sees a template? Generate all the machine code for all possible data types - ints, doubles, float, strings, ... Could take a lot of time. Or just be a little lazy and generate the machine code for what it requires.
I guess the latter option is the better solution and gets the job done.
The main point here is that compiler does not treat a template definition until it meets a certain instance of the template. (Then it can proceed, I guess, like it have a usual class, which is a specific case of the template class, with fixed template parameters.)
The direct answer to your question is: Compiler generates machine code from users c++ code, I think this is wat is meant here by word "generate code".
The template declaration must be in header file because when compiler compiles some source, which use template it HAVE only header file (included in source with #include macro), but it NEED whole template definition. So logical conclusion is that template definition must be in header.
When you create a function and compile it, the compiler generates code for it. Many compilers will not generate code for static functions that are not used.
If you create a templated function and nothing uses the template (such as std::sort), the code for the function will not be generated.
Remember, templates are like stencils. The templates tell how to generate a class or function using the given template parameters. If the stencil is not used, nothing is generated.
Consider also that the compiler doesn't know how to implement or use the template until it sees all the template parameters resolved.
It won't straight away generate code. Only generates the class or template code when it comes across an instantiation of that template. That is, if you are actually creating an object of that template definition.
In essence, templates allow you abstract away from types. If you need two instantiations of the template class for example for an int and a double the compiler will literally create two of these classes for you when you need them. That is what makes templates so powerful.
Your C++ is read by the compiler and turned into assembly code, before being turned in machine code.
Templates are designed to allow generic programming. If your code doesn't use your template at all, the compiler won't generate the assembly code associated. The more data types you associate your template with in your program, the more assembly code it will generate.
When we have a templated class (or function) that has a non-type template parameter, how are the versions generated by the compiler? Surely it doesn't create a version for every possible value of N
Suppose something like std::array<T,N> ?
I am trying to write my own templated function with a size_t template parameter, and I am trying to figure out whether/how I need to explicitly instantiate versions I will use
(I have split the program across different translation units)
I have a templated function which is something like
template <size_t N>
std::bitset<N> f(std::bitset<N> A){}
I put the declaration into a header file and definition into a .cpp file. It doesn't generate the correct version. So I tried to explicitly instantiate it like
template std::bitset<10> f<10>(std::bitset<10> A);
but I get the error that "error: explicit instantiation of .... but no definition available"
Neither the standard library nor you need to explicitly instantiate any template specialization, whether it has non-type template parameters or not.
Any specialization of the template which is used by the user in a way that requires a definition for the specialization to exist will cause it to automatically be implicitly instantiated if a definition for the entity which is instantiated is available. The condition should normally always be fulfilled since it is standard practice to place definitions of templated entities into the header file along with their initial declaration, so that they are accessible in all translation units using them.
So std::array<T,N> will be implicitly instantiated for all pairs of types T and values N which are actually used by the translation unit.
The only situation where explicit instantiation is required is if you intent to separate the definitions of template members into a single translation unit instead of the header file where they would be available for all translation units to implicitly instantiate. But you can only do that in the first place if you know all possible values for the template parameters, and you would then need to explicitly instantiate all of them, which for most non-type template parameters is not feasible.
Aside from that explicit instantiation may be used as an optimization of compilation time by avoiding implicitly instantiating template specializations in every translation unit where they are used. That makes sense only for specializations which you know will be often used. For example some standard library implementations apply it to std::string, which is the specialization std::basic_string<char>. It doesn't really make sense to apply it to e.g. std::array.
I believe that there are 4 situations where my question may have different answers. These situations are sorted by member vs. non-member functions and within vs. without a library.
Non-member function within a library
Suppose that I have defined a template function func in header func.h.
// func.h
template <typename T>
int func(T t){
//definition
}
I #include "func.h" in two cpp files of the same project/library/executable and call
//a.cpp
#include "func.h"
//stuff
int m = func<int>(3);
//stuff
and
//b.cpp
#include "func.h"
//stuff
int n = func<int>(27);
//stuff
My understanding is that these two cpp files should compile into their own object files. In what object file is func<int> instantiated? Why will the One Definition Rule not be violated? For this basic application of templates, is there any benefit to explicitly instantiating func<int> separate from its use?
Member function within a library
Suppose that func is instead a member function of some class Func.
// func.h
class Func {
template <typename T>
int func(T t){
//definition
}
};
Where will func be instantiated? Will func<int> be linked to or placed inline?
Member and Non-Member functions across libraries
Suppose that a.cpp and b.cpp are in different libraries that are compiled separately and later linked into an executable. Will the different libraries their own definitions of func<int>? At link time, why will the One Definition Rule not be violated?
Note: There is a related question of the same title here, but in a specific situation with one cpp file.
In what object file is func<int> instantiated?
In every object file (aka translation unit) that invokes it or takes an address of it when the template definition is available.
Why will the One Definition Rule not be violated?
Because the standard says so in [basic.def.odr].13.
Also see https://en.cppreference.com/w/cpp/language/definition
There can be more than one definition in a program of each of the following: class type, enumeration type, inline function, inline variable (since C++17), templated entity (template or member of template, but not full template specialization), as long as all of the following is true...
For this basic application of templates, is there any benefit to explicitly instantiating func<int> separate from its use?
In this case you get no inlining but possibly smaller code. If you use link-time code generation, then inlining may still happen.
All your questions do not really differ when it comes to answer. Regardless of template function being a member or a free function, it is going to be instantiated on first use (with given types) in each compilation unit (.cpp file).
From the compiler standpoint, ODR is not violated here, since there are no two prohibited definitions of templated function. Standard explicitly allows several definitions of templated functions.
Yet your intuition that you end up with definition of instantiated function twice in object files is correct. Luckily, ODR doesn't apply at this point. Instead, those definitions are generated with so-called 'weak' symbol - telling linker that those two symbols are identical, and it is free to pick one (or none and perform link-time optimization!)
I'm reading a book about how templates work, and I'm having difficulty understanding this explanation of templates.
It says
When the compiler sees the definition of a template, it does not generate code. It generates code only when we instantiate a specific instance of the template. The fact that code is generated only when we use a template (and not when we define it) affects how we organize our source code and when errors are detected...To generate an instantiation, the compiler needs to have the code that defines a function template or class template member function. As a result, unlike non-template code, headers for templates typically include definitions as well as declarations.
What exactly does it mean by "generate code"? I don't understand what is different when you compile function templates or class templates compared to regular functions or classes.
The compiler generates the code for the specific types given in the template class instantiation.
If you have for instance a template class declaration as
template<typename T>
class Foo
{
public:
T& bar()
{
return subject;
}
private:
T subject;
};
as soon you have for example the following instantiations
Foo<int> fooInt;
Foo<double> fooDouble;
these will effectively generate the same linkable code as you would have defined classes like
class FooInt
{
public:
int& bar()
{
return subject;
}
private:
int subject;
}
and
class FooDouble
{
public:
double& bar()
{
return subject;
}
private:
double subject;
}
and instantiate the variables like
FooInt fooInt;
FooDouble fooDouble;
Regarding the point that template definitions (don't confuse with declarations regardless of templates) need to be seen with the header (included) files, it's pretty clear why:
The compiler can't generate this code without seeing the definition. It can refer to a matching instantiation that appeared first at linking stage though.
What does a non-template member function have that allows for it to
be defined outside of the header that a template function doesn't
have?
The declaration of a non-template class/member/function gives a predefined entry point for the linker. The definition can be drawn from a single implementation seen in a compiled object file (== .cpp == compilation unit).
In contrast the declaration of a templated class/member/function might be instantiated from arbitrary compilation units given the same or varying template parameters. The definition for these template parameters need's to be seen at least once. It can be either generic or specialized.
Note that you can specialize template implementations for particular types anyway (included with the header or at a specific compilation unit).
If you would provide a specialization for your template class in one of your compilation units, and don't use your template class with types other than specialized, that also should suffice for linking it all together.
I hope this sample helps clarifying what's the difference and efforts done from the compiler.
A template is a pattern for creating code. When the compiler sees the definition of a template it makes notes about that pattern. When it sees a use of that template it digs out its notes, figures out how to apply the pattern at the point where it's being used, and generates code according to the pattern.
What is the compiler suppose to do when it sees a template? Generate all the machine code for all possible data types - ints, doubles, float, strings, ... Could take a lot of time. Or just be a little lazy and generate the machine code for what it requires.
I guess the latter option is the better solution and gets the job done.
The main point here is that compiler does not treat a template definition until it meets a certain instance of the template. (Then it can proceed, I guess, like it have a usual class, which is a specific case of the template class, with fixed template parameters.)
The direct answer to your question is: Compiler generates machine code from users c++ code, I think this is wat is meant here by word "generate code".
The template declaration must be in header file because when compiler compiles some source, which use template it HAVE only header file (included in source with #include macro), but it NEED whole template definition. So logical conclusion is that template definition must be in header.
When you create a function and compile it, the compiler generates code for it. Many compilers will not generate code for static functions that are not used.
If you create a templated function and nothing uses the template (such as std::sort), the code for the function will not be generated.
Remember, templates are like stencils. The templates tell how to generate a class or function using the given template parameters. If the stencil is not used, nothing is generated.
Consider also that the compiler doesn't know how to implement or use the template until it sees all the template parameters resolved.
It won't straight away generate code. Only generates the class or template code when it comes across an instantiation of that template. That is, if you are actually creating an object of that template definition.
In essence, templates allow you abstract away from types. If you need two instantiations of the template class for example for an int and a double the compiler will literally create two of these classes for you when you need them. That is what makes templates so powerful.
Your C++ is read by the compiler and turned into assembly code, before being turned in machine code.
Templates are designed to allow generic programming. If your code doesn't use your template at all, the compiler won't generate the assembly code associated. The more data types you associate your template with in your program, the more assembly code it will generate.
I just spent about an 20 minutes trying to figure out why some template methods of mine passed compilation but not linkage.
Turns out I needed to explicitly declare my template method.
It was something of this kind :
class Test {
template<class Source> void Save(Source& obj);
};
Then I would use it like this somewhere :
Test t;
ClassDerivedFromInterface obj;
t.Save(obj);
It compiled fine but didn't link. Until I added :
template void Test::Save(ClassDerivedFromInterface);
I would like to understand in which case an explicit declaration is necessary.
Thanks
In a nutshell, you need to have the entire body (the definition) of a template function visible to the translation unit that instantiates the template. So when you say t.Save(obj);, that translation unit should have access to the definition of Save. Usually you achieve this by including the definitions of function templates in the header file itself.
The reason for this is that templates aren't ordinary code that gets compiled and can later be linked at will. Rather, templates are a code generation tool that generates the necessary code on demand - an automatic version of copy/paste followed by search-and-replace, if you will.
Therefore, the actual compilable code for your function Save(ClassDerivedFromInterface&) doesn't come into existence until you write that line. If only the declaration of the function template is visible, then the template only produces the declaration of the concrete function, but not its body, and so at link time you notice that the function is missing.
To recap, templates themselves cannot be compiled, it is only their concrete instances that can, and you have to pay attention to ensure that the concrete instances are always available when you instantiate them. Explicit instantiation as you have it works and allows you to package a few specific instances into a separate TU, but generally that's hard to maintain and not scalable, and there are other drawbacks to explicit instantiation that you avoid when you let the compiler instantiate implicitly. So usually it's best to package your entire definitions into the header file.
You will need to explicitly declare the template if the template source is not visible at the time of compilation. This link covers it pretty well, also an awesome site in general:
C++ FAQ 35.13
You need explicit declaration when your template definitions are not accesible from the code that is using them, consider the following:
template.h - template declarations
template.cpp - template definitions
main.cpp - template usage
template.cpp is not included in main.cpp and thus unreachable by the template user so you need explicit declarations.
But if your structure is:
template.h - template declarations and definitions
main.cpp - template usage
the template declarations are reachable by the template user so you don't need explicit declarations.
The compiler needs to know what types you're going to be using with your template. If you create a template class and then use it with, for example, int, char, and double, then the compiler will create methods for the template for those types. If you compile the template method in a separate compilation unit from where you use it, the compiler will not instantiate your template for the type you need. But if you explicitly instantiate the template, the compiler will create whatever you tell it to.