I have templated functions in my C++11 Xcode project and some of them have specializations. However, I have discovered that the specializations only get called in debug builds; if I build in release, they are ignored.
I have successfully created a very simple example:
special.h
#include <cstdio>
struct special
{
template<typename T>
void call(const T&) { puts("not so special"); }
};
special.cpp
#include "special.h"
#include <string>
template<>
void special::call(const std::string&) { puts("very special"); }
main.cpp
#include "special.h"
#include <string>
int main()
{
std::string str = "hello world";
special s;
s.call(123);
s.call(str);
}
You can download the project (until somewhere this summer of 2013 at least) to reproduce the issue if you don't want to create it yourself. First run the project with the debug configuration, then run it again in release. The output that I expect is:
not so special
very special
And this is indeed what I get with the Debug build configuration. However, with Release, I get this:
not so special
not so special
Which means the specialized implementation of special::call in special.cpp was ignored.
Why is the result inconsistent? What should I do to ensure that the specialized function is called in release builds?
Your program has UB. An explicit specialisation or at least its declaration must be visible before being used. [temp.expl.spec]§6:
If a template, a member template or a member of a class template is
explicitly specialized then that specialization shall be declared
before the first use of that specialization that would cause an
implicit instantiation to take place, in every translation unit in
which such a use occurs; no diagnostic is required.
Add this declaration to special.h:
template<>
void special::call(const std::string&);
Alternatively, you can put the specialistation itself into the header. However, as a specialisation is no longer a template, it follows normal function rules and must be marked inline if placed in a header.
Also, be careful that function template specialisations have rather specific behaviour, and it's generally better to use overloads than specialisations. See Herb Sutter's article for details.
You violated the one definition rule (ODR). So what happens exactly? In main.cpp there is no specialization known for special::call<string>. Therefore the compiler generates an instantiation of the template into that translation unit (TU) that outputs "not so special". In special.cpp there is a full specialization declared and defined, so the compiler puts that definition into the other translation unit. So you have two different definitions of the very same function in two different translation units, which is a violation of the ODR which means it is undefined behavior.
In theory, the outcome can be anything. A compiler error, a crash, a silent online order for a pizza, anything. Even different behavior in debug and release compiles.
In practice, I guess the following happens: When linking the debug build, the Linker sees the same symbol defined twice in the two TUs, which is allowed only for templates and inline functions. Because of the ODR it may assume that both definitions are equivalent and picks the one from special.cpp, so you get by coincidence the behavior you expect.
During release build, the compiler inlines the call to special::call<string> during the compilation of main.cpp, so you get the only behavior seen in that TU: "not so special".
So how can you fix this?
In order to have only one definition for that specialization, you have to define it in one TU as you did, but you have to declare that there is a full specialization in any other TU, meaning declare that the specialization exists in the header special.h:
// in special.h
template<>
void special::call(const std::string&);
Or, which is seen more often, define it in the header, so it's seen in every TU. Since fully specialized function templates are normal functions, you will have to define it inline:
// in special.h
template<>
inline void special::call(const std::string&)
{
puts("very special");
}
Related
Suppose I have two .cpp files file1.cpp and file2.cpp:
// file1.cpp
#include <iostream>
inline void foo()
{
std::cout << "f1\n";
}
void f1()
{
foo();
}
and
// file2.cpp
#include <iostream>
inline void foo()
{
std::cout << "f2\n";
}
void f2()
{
foo();
}
And in main.cpp I have forward declared the f1() and f2():
void f1();
void f2();
int main()
{
f1();
f2();
}
Result (doesn't depend on build, same result for debug/release builds):
f1
f1
Whoa: Compiler somehow picks only the definition from file1.cpp and uses it also in f2(). What is the exact explanation of this behavior?.
Note, that changing inline to static is a solution for this problem. Putting the inline definition inside an unnamed namespace also solves the problem and the program prints:
f1
f2
This is undefined behavior, because the two definitions of the same inline function with external linkage break C++ requirement for objects that can be defined in several places, known as One Definition Rule:
3.2 One definition rule
...
There can be more than one definition of a class type (Clause 9), enumeration type (7.2), inline function with external linkage (7.1.2), class template (Clause 14),[...] in a program provided that each definition appears in a different translation unit, and provided the definitions satisfy the following requirements. Given such an entity named D defined in more than one translation unit, then
6.1 each definition of D shall consist of the same sequence of tokens; [...]
This is not an issue with static functions, because one definition rule does not apply to them: C++ considers static functions defined in different translation units to be independent of each other.
The compiler may assume that all definitions of the same inline function are identical across all translation units because the standard says so. So it can choose any definition it wants. In your case, that happened to be the one with f1.
Note that you cannot rely on the compiler always picking the same definition, violating the aforementioned rule makes the program ill-formed. The compiler could also diagnose that and error out.
If the function is static or in an anonymous namespace, you have two distinct functions called foo and the compiler must pick the one from the right file.
Relevant standardese for reference:
An inline function shall be defined in every translation unit in which it is odr-used and shall have exactly
the same definition in every case (3.2). [...]
7.1.2/4 in N4141, emphasize mine.
As others have noted, the compilers are in compliance with the C++ standard because the One definition rule states that you shall have only one definition of a function, except if the function is inline then the definitions must be the same.
In practice, what happens is that the function is flagged as inline, and at linking stage if it runs into multiple definitions of an inline flagged token, the linker silently discards all but one. If it runs into multiple definitions of a token not flagged inline, it instead generates an error.
This property is called inline because, prior to LTO (link time optimization), taking the body of a function and "inlining" it at the call site required that the compiler have the body of the function. inline functions could be put in header files, and each cpp file could see the body and "inline" the code into the call site.
It doesn't mean that the code is actually going to be inlined; rather, it makes it easier for compilers to inline it.
However, I am unaware of a compiler that checks that the definitions are identical before discarding duplicates. This includes compilers that otherwise check definitions of function bodies for being identical, such as MSVC's COMDAT folding. This makes me sad, because it is a reall subtle set of bugs.
The proper way around your problem is to place the function in an anonymous namespace. In general, you should consider putting everything in a source file in an anonymous namespace.
Another really nasty example of this:
// A.cpp
struct Helper {
std::vector<int> foo;
Helper() {
foo.reserve(100);
}
};
// B.cpp
struct Helper {
double x, y;
Helper():x(0),y(0) {}
};
methods defined in the body of a class are implicitly inline. The ODR rule applies. Here we have two different Helper::Helper(), both inline, and they differ.
The sizes of the two classes differ. In one case, we initialize two sizeof(double) with 0 (as the zero float is zero bytes in most situations).
In another, we first initialize three sizeof(void*) with zero, then call .reserve(100) on those bytes interpreting them as a vector.
At link time, one of these two implementations is discarded and used by the other. What more, which one is discarded is likely to be pretty determistic in a full build. In a partial build, it could change order.
So now you have code that might build and work "fine" in a full build, but a partial build causes memory corruption. And changing the order of files in makefiles could cause memory corruption, or even changing the order lib files are linked, or upgrading your compiler, etc.
If both cpp files had a namespace {} block containing everything except the stuff you are exporting (which can use fully qualified namespace names), this could not happen.
I've caught exactly this bug in production multiple times. Given how subtle it is, I do not know how many times it slipped through, waiting for its moment to pounce.
POINT OF CLARIFICATION:
Although the answer rooted in C++ inline rule is correct, it only applies if both sources are compiled together. If they are compiled separately, then, as one commentator noted, each resulting object file would contain its own 'foo()'. HOWEVER: If these two object files are then linked together, then because both 'foo()'-s are non-static, the name 'foo()' appears in the exported symbol table of both object files; then the linker has to coalesce the two table entries, hence all internal calls are re-bound to one of the two routines (presumably the one in the first object file processed, since it is already bound [i.e the linker would treat the second record as 'extern' regardless of binding]).
In the code below, foo should be a function accessible by anyone, but foo_helper should not, which is why I've put it in an anonymous namespace. Obviously I'm leaving out include guards and includes in this example, but they are there.
foo.h:
namespace
{
void foo_helper() {}
template <typename T, typename... Tail>
void foo_helper(T head, Tail... tail)
{
bar(head);
foo_helper(tail...);
}
}
void foo();
template <typename... Args>
void foo(Args... args)
{
before();
foo_helper(args...);
after();
}
foo.cpp:
void foo() {}
The problem is that in order for foo_helper's variadic template to work, it needs to have that initial version with no argument. But, this forces me to define a non-template function is a header file, which would break after including this file in multiple source files. I cannot move the definition of foo_helper to a source file because it is in an anonymous namespace, since it is not supposed to be accessible.
Is there a way to solve this issue?
inline void foo_helper() {};
solves your problem.
inline mostly means "conflicting definitions of this function are to be discarded, and one of the versions kept".
It also non-bindingly suggests "inlining" in a vague way (in that the standard doesn't really cover what inlining is). Compilers may or may not pay attention to that suggestion.
Note that an anonymous namespace does not "make it unusable" or whatever. Anonymous namespaces are designed to block linker collisions, and that is about it. Create a namespace called details and ... well, trust users not to go and poke inside.
Using an anonymous namespace in a header is a very bad idea.
If there is an inline function (or template function) within another header file that accesses a symbol or function within the anonymous namespace, you are almost certainly going to have an ODR (one definition rule) violation. That is where the same object, function, etc has two definitions that differ, and isn't allowed to.
For example:
inline void bob() {
foo(1,2,3);
}
if that is #included in two different .cpp files, you just made an ill-formed program (no diagnostic required).
Often such ill formed programs "behave the way you expect", but sometimes they do not. As an example, if somewhere along the line you get a static local variable whose existence depends on ODR violation, you can have multiple compilation units disagree asto which one exists and what its properties are.
In a more general sense, the link order of your program could change its behavior, as different definitions are "chosen" (with possibly extremely subtle differences). Or the phase of the moon could do the same.
ODR violations are surprisingly benign, until they byte you with non-local bugs that are hard to track down.
I'll start with an aside: using an anonymous namespace here doesn't serve your purpose. Since you're defining it in a header file, it's not protected at all: it will still be in scope in any file that includes your header. Additionally, since you've defined it in an anonymous namespace, a separate copy of the function will be emitted in each translation unit that uses it, and the linker can't collapse them. If you really want it to be private, I'm not on the up-and-up of the best C++ style these days, so perhaps someone else will correct me, but I would be inclined to use a private namespace:
namespace my_stuff {
void foo_helper();
}
void foo() {
my_stuff::foo_helper();
}
As Yakk indicated, you could use an inline function, and that will allow the compiler to collapse the definitions into one. In modern practice, there shouldn't really be another reason to avoid the inline keyword, because compilers will nowadays decide on their own whether or not to inline functions rather than listening to the hints you give.
Since you have defined the function in an anonymous namespace, as I mentioned above, you actually don't need to do anything else to avoid linker errors if you retain that. The downside to this approach is that you will have separate copies of foo_helper() in each translation unit and those can't be merged by the linker.
There are other gymnastics you could do, mostly involving sizeof..., but I don't think those are ideal.
I'm looking for an overview or description of what GCC does differently than MSVC for compile + link of template classes with specializations. For example, this type of thing works on GCC but not MSVC:
// Base.h
template <typename T> struct Base {
template <class G> QString makeTitle(const G* obj){obj->CompilerError();}
};
// in Foo.cpp
template <> template <class G> QString Base<T_1>::makeTitle(const G* obj) { return mystr(); }
void SomeFunc() {
std::cout<< Base<T_1>().makeTitle<myclass>() ;
}
and the solution tends to be that I must declare the specialization in Base.h before using it, or on windows there is a link error. How/why does MSVC implicitly instantiate differently, and how/why is GCC robust to the specialization being declared in some cpp file?
related question that notes the general 'declare before using' requirement: Template specialization - different behaviour between MSVC and GCC/MinGW
The first thing is that your code is in violation of the ODR rule if any translation unit includes the header and causes the definition of the specialization without the compiler seeing a declaration. Since that is undefined behavior the fact that one compiler accepts it and the other rejects it is well within reason.
The correct code, as you already figured out is to provide the declaration of the specialization, and that will work in any compiler.
As of why it seems to work or even why does it actually work in gcc, this is most probably a matter of how the code is generated and the linker processes the object files. In particular in gcc the compiler will generate the specialization in the translation unit that needs it (and does not see your own specialization), but it will be marked as a weak symbol. The gcc linker will accept a symbol being multiply defined if all but [at most] one definitions are weak, leaving the strong symbol in the final executable.
Lets say you have simple template function (not class member for the sake of simplicity) with type specific specialization in the same .h file...
template <class TYPE>
void some_function(TYPE& val)
{
// some generic implementation
}
template <>
inline void some_function<int>(int& val)
{
// some int specific implementation
}
Unless you explicitly direct you compiler to inline the specialization (inline keyword) you will get linking error if .h file is included more than once (at least I do in Visual C++ 2008).
We all know that inline is just a suggestion to the compiler, which it can ignore. In this particular case is compiler allowed to ignore this suggestion and let linker to fail?
If you don't use inline, then the same function gets compiled with extern linkage into multiple .obj files, which causes the linker to throw a duplicate symbol error.
This is independent of whether the compiler actually compiles your function inline, since it could treat it the same as a static function and make each implementation private to each compilation unit. However, you can't use static for this purpose since it means something else on member functions, so inline is your only choice.
You are misunderstanding the meaning of the often-mentioned "ignore inline" possibility.
No compiler is ever allowed to ignore the inline specifier used in function declaration and the consequences this specifier has with respect to One Definition Rule (ODR).
When someone says that compiler are allowed to "ignore inline", it only means that compilers are not required to actually inline the calls to the function in question. To "ignore inline" means to generate an ordinary (non-inlined) function call to an inline function.
In any case, even if the compiler decided to always generate ordinary calls to an inline function (i.e. to always "ignore inline"), it is still required to treat the function as inline for the purposes of ODR. How the compiler is going to do it is the problem of the compiler. You are not supposed to worry about it.
In your original example you should not not get any linker errors.
This is defined by the standard and the compiler is totally compliant in this regard, from the looks of it. The linkage is all you are after. Implicit template instantiations have 'special' linkage, as inline functions do. There is also static (keyword), which has been deprecated in favor of anonymous namespaces:
namespace {
…declarations…
}
So yes, this specialization (in your example) has the same linkage as:
void some_other_function(int& val) {
// some int specific implementation
}
In fact, the compiler may mumble about inlining the specialization, in your example, saying they do not match. So it really is a best practice to label them both inline (or otherwise).
I believe you can explicitly declare the method as extern and then put the specialization into a .cpp. I've tried something similar in a past life with GCC, but I don't recall the exact details of how it worked. MSDN Magazine has an article on this that might help.
What you're actually seeing is the One Definition Rule (ODR) has a special case for inline functions, in that each TU may have a definition. If the function, such as your explicit int specialization, is not inline, then you will get multiple definition errors at link time. Such inline functions still have external linkage. Function templates are templates and so follow different rules. Instantiations/specializations of a function template are functions.
Using inline, as for any function, is just a hint, but you might want to apply it if the function is short (as for any function) or if you just want to keep it in the header. Here's an example without inline:
Header file:
template<class TYPE>
void some_function(TYPE& val) {
// some generic implementation
}
template<>
void some_function<int>(int& val);
Implementation (.cpp) file:
template<>
void some_function<int>(int& val) {
// some int specific implementation
}
Suppose I have fileA.h which declares a class classA with template function SomeFunc<T>(). This function is implemented directly in the header file (as is usual for template functions). Now I add a specialized implementation of SomeFunc() (like for SomeFunc<int>()) in fileA.C (ie. not in the header file).
If I now call SomeFunc<int>() from some other code (maybe also from another library), would it call the generic version, or the specialization?
I have this problem right now, where the class and function live in a library which is used by two applications. And one application correctly uses the specialization, while another app uses the generic form (which causes runtime problems later on). Why the difference? Could this be related to linker options etc? This is on Linux, with g++ 4.1.2.
It is an error to have a specialization for a template which is not visible at the point of call. Unfortunately, compilers are not required to diagnose this error, and can then do what they like with your code (in standardese it is "ill formed, no diagnostic required").
Technically, you need to define the specialization in the header file, but just about every compiler will handle this as you might expect: this is fixed in C++11 with the new "extern template" facility:
extern template<> SomeFunc<int>();
This explicitly declares that the particular specialization is defined elsewhere. Many compilers support this already, some with and some without the extern.
Have you added a prototype with parameters to your header file?
I mean is there somewhere in fileA.h
template<> SomeFunc<int>();
If not that's probably the reason.
I had the same problem with gcc4, here is how i solved it. It was more simple a solution than what i was lead to believe by previous comments. The previous posts ideas were correct but their syntax didn't work for me.
----------header-----------------
template < class A >
void foobar(A& object)
{
std::cout << object;
}
template <>
void foobar(int);
---------source------------------
#include "header.hpp"
template <>
void foobar(int x)
{
std::cout << "an int";
}
Per the specs, your specialized function template should never be called outside fileA.C, unless you export the template definition, which no compiler (except Comeau) currently supports (or has it planned for the forseeable future).
On the other hand, once the function template is instantiated, there is a function visible to the compiler that is no longer a template. GCC may re-use this definition across different compiler units because the standard states that each template shall only be instantiated once for a given set of type arguments [temp.spec]. Still, since the template is not exported, this should be limited to the compilation unit.
I believe that GCC may expose a bug here in sharing its list of instantiated templates across compilation units. Normally, this is a reasonable optimization but it should take function specializations into account which it doesn't seem to do correctly.
In Microsoft C++, I did an experiment with inline functions. I wanted to know what would happen if I defined incompatible versions of a function in different sources. I got different results depending on whether I was using a Debug build or a Release build. In Debug, the compiler refuses to inline anything, and the linker was linking the same version of the function no matter what was in scope in the source. In Release, the compiler inlined whichever version had been defined at the time, and you got differing versions of the function.
In neither case were there any warnings. I kind of suspected this, which is why I did the experiment.
I assume that template functions would behave the same, as would other compilers.
As Anthony Williams says, the extern template construct is the correct way to do this, but since his sample code is incomplete and has multiple syntax errors, here's a complete solution.
fileA.h:
namespace myNamespace {
class classA {
public:
template <class T> void SomeFunc() { ... }
};
// The following line declares the specialization SomeFunc<int>().
template <> void classA::SomeFunc<int>();
// The following line externalizes the instantiation of the previously
// declared specialization SomeFunc<int>(). If the preceding line is omitted,
// the following line PREVENTS the specialization of SomeFunc<int>();
// SomeFunc<int>() will not be usable unless it is manually instantiated
// separately). When the preceding line is included, all the compilers I
// tested this on, including gcc, behave exactly the same (throwing a link
// error if the specialization of SomeFunc<int>() is not instantiated
// separately), regardless of whether or not the following line is included;
// however, my understanding is that nothing in the standard requires that
// behavior if the following line is NOT included.
extern template void classA::SomeFunc<int>();
}
fileA.C:
#include "fileA.h"
template <> void myNamespace::classA::SomeFunc<int>() { ... }
Brandon: that's what I thought - the specialized function should never be called. Which is true for the second application I mentioned. The first app, however, clearly calls the specialized form even though the specialization is not declared in the header file!
I mainly seek enlightenment here :-) because the first app is a unit test, and it's unfortunate to have a bug that doesn't appear in the test but in the real app...
(PS: I have fixed this specific bug, indeed by declaring the specialization in the header; but what other similar bugs might still be hidden?)
#[anthony-williams],
are you sure you're not confusing extern template declarations with extern template instantiations? From what I see, extern template may only be used for explicit instantiation, not for specialization (which implies implicit instantiation). [temp.expl.spec] doesn't mention the extern keyword:
explicit-specialization:
template < > declaration
Unless the specialized template function is also listed in the header file, the other application will have no knowledge of the specialized version. The solution is the add SomeFunc<int>() to the header as well.