Why did C++03 require template parameters to have external linkage? - c++

Background
In C++03, symbols used as template arguments must have external linkage; this restriction was removed in C++11, as explored in this previous question:
In C++03, template arguments could not have internal linkage:
[C++03: 14.6.4.2/1]: For a function call that depends on a template parameter, if the function name is an unqualified-id but not a template-id, the candidate functions are found using the usual lookup rules (3.4.1, 3.4.2) except that:
For the part of the lookup using unqualified name lookup (3.4.1), only function declarations with external linkage from the template definition context are found.
For the part of the lookup using associated namespaces (3.4.2), only function declarations with external linkage found in either the template definition context or the template instantiation context are found.
[..]
This was changed (issue #561: "Internal linkage functions in dependent name lookup") in C++11:
[C++11: C.2.6]: 14.6.4.2
Change: Allow dependent calls of functions with internal linkage
Rationale: Overly constrained, simplify overload resolution rules.
resulting in:
[C++11: 14.6.4.2/1]: For a function call that depends on a template parameter, the candidate functions are found using the usual lookup rules (3.4.1, 3.4.2, 3.4.3) except that:
For the part of the lookup using unqualified name lookup (3.4.1) or qualified name lookup (3.4.3), only function declarations from the template definition context are found.
For the part of the lookup using associated namespaces (3.4.2), only function declarations found in either the template definition context or the template instantiation context are found.
[..]
(Spot the missing "with external linkage" qualification.)
Issue #561 ("Internal linkage functions in dependent name lookup"), the proposal that led to the restriction being removed in C++11, asks:
Furthermore, is it really necessary to exclude internal linkage functions from the lookup? Doesn't the ODR give implementations sufficient latitude to handle this case without another wrinkle on name lookup?
With the later answer:
The consensus of the group was that [..] internal-linkage functions should be found by the lookup (although they may result in errors if selected by overload resolution).
Question
What was the original practical rationale for the restriction?
It seems like there must have been one, since the original standard wording went out of its way to limit lookup to symbols with external linkage.
Is it just that "[internal-linkage functions] may result in errors if selected by overload resolution", and that through the 2000s opinion shifted over how important this was? Or did something else change, perhaps as an indirect consequence of new wording elsewhere for a different C++11 feature?

I would suspect it's related to the infamous export template feature of C++98. Think about it. Once you allow the possibility of template definitions appearing in separate translation units, but still not truly compilable until the template arguments are specified (i.e., the template is instantiated), you get into this twilight zone where the TU with the definition of the template and the TU with the instantiation have to obey linker-visibility rules (i.e., separation model) while sharing their contexts in terms of overload resolution. The solution to that problem is to allow only functions with external linkage in the dependent name lookups.
Here's an example. One of the lesser known "feature" of exported templates is that you could have, in the template's TU, some functions or classes with internal linkage (i.e., marked static or in an un-named namespace). What if the TU with the instantiation has a internal-linkage function too, one that would be ambiguous with or possibly superceded by the one in the template's TU? That's a bit of a surreal problem, I know, it's the bizarro world of exported templates. The only way to avoid very surprising behavior is to rule out all internal-linkage functions from the lookups. Consider also that no-one had a clear idea of how to actually implement exported templates, and it would probably seem even more impossible to implement without this restriction.
And so, once exported templates are out, the restriction on dependent name lookups seems clearly useless and it was taken out without much debate. At least, that makes perfect sense to me, but, of course, it's speculation.
Here's a concrete example:
// in exptemp.h
export template <typename T> bool is_valid(T value);
// in exptemp.cpp
namespace {
bool is_space(char c) {
return (c == ' ') || (c == '\t');
};
};
template <typename T>
bool is_valid(T value) {
return is_space(value);
};
// in test.cpp
#include "exptemp.h"
namespace {
bool is_space(char c) {
return (c == ' ') || (c == '\t') || (c == '\n');
};
};
int main() {
char c = '\n';
return is_valid(c); // will this return 0 or 1 ?!?!?
};

As far as I know, it's purely historical -- it seems to have originally been banned because cfront's name mangling wasn't sufficient to handle it correctly.
At one point, Anthony Williams wrote a paper proposing that it would be allowed, and telling how to do it -- but AFAIK, that paper has never been accepted nor its requirements edited into the standard. I suspect that's as much a matter of timing as anything else though. It was proposed in 2001, so what they were working on at the time (C++ 2003) wasn't intended to add much new material, and by the time they started working in earnest on C++11, it seems to have been mostly forgotten.

Related

Argument-dependent lookup of dependent names

This description on cppreference.com says that
The lookup of a dependent name used in a template is postponed until the template arguments are known, at which time [...] ADL examines function declarations with external linkage that are visible from either the template definition context or the template instantiation context.
Contrary to this the following code snippet compiles fine with three compilers (MSVC, clang, gcc):
template <class T>
void CallFoo ()
{
Foo (T ());
}
class Apple {};
int main ()
{
CallFoo<Apple> ();
}
static void Foo (Apple)
{
}
Foo is a dependent name in CallFoo: it depends on template argument T. But the function Foo is found by the compiler despite violating two of the above quoted rules.
The declaration of Foo is not visible from either the definition or instantiation of CallFoo, because it is below both.
Foo has internal linkage.
It is unlikely, that all three compilers have a bug. I might have misunderstood something. Could you elaborate on this?
In C++03, members of anonymous namespaces could have external linkage despite being unnameable in other translation units. It was therefore thought permissible to preclude actual static functions from dependent ADL. In C++11, anonymous namespaces impose internal linkage, so the restriction became unreasonable. However, despite implementations adopting the new behavior and an issue being filed immediately in 2011 (as noted in the comments), the wording remained in two places until N4810 in March 2019.
As for the placement of the function, this is an artifact of functions having multiple points of instantiation, including the end of any translation unit that instantiates them (with slight adjustments for modules in C++20); if instantiating the function template produces different results for different choices, the program is ill-formed, no diagnostic required (as also noted in the comments).

Parameter type lookup for out-of-namespace function definitions

The software project I lead occasionally uses nested namespaces as a means of indicating which parts of the library are intended to be internal and should not be considered user-facing. For example:
namespace project::detail {
// Hands off; library implementation internals
class Internal {};
int important_number(Internal internals);
}
Before C++17, the indentation would become somewhat dreadful (depending on the source code formatting), so we opted for defining functions out-of-namespace:
int
project::detail::important_number(Internal const internals)
{ ... }
Observe that although the type Internal was declared and defined in the project::detail nested namespace, it is not required to fully qualify the parameter type (i.e.):
int
project::detail::important_number(/*unnecessary*/ project::detail::Internal)
I have scoured en.cppreference.com and bits of the C++14 standard and have not found the place(s) that support the abbreviated function definition (the first example above). I am assuming the relevant rules did not change for C++17. Can anyone tell me which rules of the C++14/17 standard apply here--i.e. where does the standard support non-fully-qualified parameter types for out-of-namespace function definitions?
You're looking for unqualified name lookup and it can be found in the standard under basic.lookup.unqual
In the definition of a function that is a member of namespace N, a
name used after the function’s declarator-id shall be declared before
its use in the block in which it is used or in one of its enclosing
blocks or shall be declared before its use in namespace N or, if
N is a nested namespace, shall be declared before its use in one of
N’s enclosing namespaces.

Template dependent name resolution should not find declarations with no linkage?

In the c++ standard [temp.point] it is written:
The instantiation context of an expression that depends on the
template arguments is the set of declarations with external linkage
declared prior to the point of instantiation of the template
specialization in the same translation unit.
Then in [temp.dep.candidate]:
For the part of the lookup using associated namespaces
([basic.lookup.argdep]), only function declarations found in either
the template definition context or the template instantiation context
are found.
Does it means that the following code should fail:
namespace A{
struct S{};
}
template<class T>
void g(T a){
f(a); //f will be found by argument dependent lookup
}
namespace A{
static void f(S); //but f doesn't have external linkage
}
void test(A::S i){
g(i);
}
//point of instantiation of g
//A::f(S) doesn't have external linkage
//=> so it's not in the instantiation context of template g ??
This code actually compiles, so what does this standard paragraph mean?
This is a defect in the standard. Originally addressed in core issue 561, where the committee judged that
Notes from the April, 2006 meeting:
The consensus of the group was [..] that internal-linkage functions should be found by the lookup (although they may result in errors if selected by overload resolution).
Unfortunately the corresponding fix was insufficient, as elaborated in core issue 1258:
C++11 expanded the lookup rules for dependent function calls (17.7.4.2 [temp.dep.candidate] paragraph 1 bullet 2) to include functions with internal linkage; previously only functions with external linkage were considered. However, 17.7.4.1 [temp.point] paragraph 6 still says,
The instantiation context of an expression that depends on the template arguments is the set of declarations with external linkage declared prior to the point of instantiation of the template specialization in the same translation unit.
Presumably this wording was overlooked and should be harmonized with the new specification.
That is, the previous wording of your second quoted paragraph was
For the part of the lookup using associated namespaces (3.4.2), only function declarations with external linkage
found in either the template definition context or the template instantiation context are found.
.. which was amended for C++11, but that change missed your first quote, making it rather pointless. The intent is that functions with internal linkage are not discriminated.

Has the C++ standard committee considered templated namespaces?

Namespaces are in many was like classes with no constructors, no destructors, no inheritance, final, and only static methods and members. After all, this kind of classes can essentially be used only the way namespaces are used: a named scope for declarations and definitions.
... except that the above is not true, since classes can be templated - and namespaces cannot. There have been a couple of questions here on the site similar to "can I template a namespace", but what I'd like to know is - has the C++ standard committee ever considered a proposal to make namespaces templatable? If it has, was the proposal rejected? If it was, what were the reasons?
The inability to have a template namespace is actually just one way in which they differ from classes. Others would be things like new namespace, and sizeof (namespace) - how could a compiler implement that, given that a namespace may extend over many compilation units?
Looking just at template namespaces... While it can at times be hard to keep up with all the proposals for new C++ features, I don't recall ever seeing one that attempted to add a feature such as you describe.
Would it ever be considered, assuming someone were to write a proposal? As Stroustrup indicates in this interview (http://www.stroustrup.com/devXinterview.html):
For C++ to remain viable for decades to come, it is essential that
Standard C++ isn't extended to support every academic and commercial
fad. Most language facilities that people ask for can be adequately
addressed through libraries using only current C++ facilities.
As you indicate yourself, what you are asking for is basically already there: just use a templated class with static members. This seems to disqualify it as a potential new feature, at least in the eyes of Stroustrup.
How would ADL work if namespaces can be templated? Are we supposed to create special template deduction rules for ADL then?
More importantly, can you justify the added complexity to the language by demonstrating a use-case that can't be filled by, just make a template struct with only static members? If a template namespace is just like a gimped template struct, that doesn't seem to be very compelling.
Also. I understand you weren't satisfied with the other questions about namespace / template hybrids, but one point in this answer seems to be relevant to your question:
Why can't namespaces be template parameters?
Possibly difficult: A namespace isn't a complete, self-contained entity. Different members of a namespace can be declared in different headers and even different compilation units.
If a namespace is a template, how will this even work? Can you still "reopen" the namespace like you can with a regular namespace? If that's allowed, then what is the point of instantiation of the namespace?
It sounds like it could potentially be extremely complicated.
Also: Will the language still be easily parsable after your proposed feature?
One of the most vexing things in C++ is the need to write template often when defining templates that refer to other templates, in order to resolve ambiguity in the grammar regarding whether < is a less than operator or a template parameter list.
3.4.5 [basic.lookup.classref]
In a class member access expression (5.2.5), if the . or -> token is immediately followed by an identifier followed by a <, the identifier must be looked up to determine whether the < is the beginning of a template argument list (14.2) or a less-than operator. The identifier is first looked up in the class of the object expression. If the identifier is not found, it is then looked up in the context of the entire postfix-expression and shall name a class template. If the lookup in the class of the object expression finds a template, the name is also looked up in the context of the entire postfix-expression and
— if the name is not found, the name found in the class of the object expression is used, otherwise
— if the name is found in the context of the entire postfix-expression and does not name a class template, the name found in the class of the object expression is used, otherwise
— if the name found is a class template, it shall refer to the same entity as the one found in the class of the object expression, otherwise the program is ill-formed.
If namespaces can be templates, don't we have to write template for them also, whenever you will refer to a template after a :: operator? For the same reason that foo::bar < 1 ... could be a namespace template bar inside of template foo with a non-type template parameter, or it could be a comparison of 1 with int foo::bar.
How do we disambiguate between that and the third possibility, foo is a namespace and bar is a regular class template inside of it`?

Can using a lambda in header files violate the ODR?

Can the following be written in a header file:
inline void f () { std::function<void ()> func = [] {}; }
or
class C { std::function<void ()> func = [] {}; C () {} };
I guess in each source file, the lambda's type may be different and therefore the contained type in std::function (target_type's results will differ).
Is this an ODR (One Definition Rule) violation, despite looking like a common pattern and a reasonable thing to do? Does the second sample violate the ODR every time or only if at least one constructor is in a header file?
This boils down to whether or not a lambda's type differs across translation units. If it does, it may affect template argument deduction and potentially cause different functions to be called - in what are meant to be consistent definitions. That would violate the ODR (see below).
However, that isn't intended. In fact, this problem has already been touched on a while ago by core issue 765, which specifically names inline functions with external linkage - such as f:
7.1.2 [dcl.fct.spec] paragraph 4 specifies that local static variables and string literals appearing in the body of an inline function with
external linkage must be the same entities in every translation unit
in the program. Nothing is said, however, about whether local types
are likewise required to be the same.
Although a conforming program could always have determined this by use
of typeid, recent changes to C++ (allowing local types as template
type arguments, lambda expression closure classes) make this question
more pressing.
Notes from the July, 2009 meeting:
The types are intended to be the same.
Now, the resolution incorporated the following wording into [dcl.fct.spec]/4:
A type defined within the body of an extern inline function is the same type in every translation unit.
(NB: MSVC isn't regarding the above wording yet, although it might in the next release).
Lambdas inside such functions' bodies are therefore safe, since the closure type's definition is indeed at block scope ([expr.prim.lambda]/3).
Hence multiple definitions of f were ever well-defined.
This resolution certainly doesn't cover all scenarios, as there are many more kinds of entities with external linkage that can make use of lambdas, function templates in particular - this should be covered by another core issue.
In the meantime, Itanium already contains appropriate rules to ensure that such lambdas' types coincide in more situations, hence Clang and GCC should already mostly behave as intended.
Standardese on why differing closure types are an ODR violation follows. Consider bullet points (6.2) and (6.4) in [basic.def.odr]/6:
There can be more than one definition of […]. Given such an entity named D defined in more than one translation unit, then each definition of D shall consist of the
same sequence of tokens; and
(6.2) - in each definition of D, corresponding names, looked up
according to [basic.lookup], shall refer to an entity defined within
the definition of D, or shall refer to the same entity, after
overload resolution ([over.match]) and after matching of partial
template specialization ([temp.over]), […]; and
(6.4) - in each definition of D, the overloaded operators referred to,
the implicit calls to conversion functions, constructors,
operator new functions and operator delete functions, shall refer to
the same function, or to a function defined within the definition of
D; […]
What this effectively means is that any functions called in the entity's definition shall be the same in all translation units - or have been defined inside its definition, like local classes and their members. I.e. usage of a lambda per se is not problematic, but passing it to function templates clearly is, since these are defined outside the definition.
In your example with C, the closure type is defined within the class (whose scope is the smallest enclosing one). If the closure type differs in two TUs, which the standard may unintentionally imply with the uniqueness of a closure type, the constructor instantiates and calls different specializations of function's constructor template, violating (6.4) in the above quote.
UPDATED
After all I agree with #Columbo answer, but want to add the practical five cents :)
Although the ODR violation sounds dangerous, it's not really a serious problem in this particular case. The lambda classes created in different TUs are equivalent except their typeids. So unless you have to cope with the typeid of a header-defined lambda (or a type depending on the lambda), you are safe.
Now, when the ODR violation is reported as a bug, there is a big chance that it will be fixed in compilers that have the problem e.g. MSVC and probably some other ones which don't follow the Itanium ABI. Note that Itanium ABI conformant compilers (e.g. gcc and clang) are already producing ODR-correct code for header-defined lambdas.