I was playing with C++17's class template argument deduction today. The first obvious idea that came to mind was passing a callable as template parameter. A callable, that's among other things a lambda, why not. Let's try that.
template<typename F> class foo
{
F f;
public:
foo(F in) : f(in) { f(); /* not very useful, admitted */ }
};
void bar() { puts("a"); }
int main()
{
auto a = foo(bar);
auto b = foo([](){ puts("b"); });
return (void) a, (void) b, 0;
}
Here is what clang (5.0, r300688) has to say about it:
warning: function '<(lambda at [source
location])>' has internal linkage but is not defined
The code compiles and certainly "works fine" but the warning suggests the compiler is not altogether happy with it.
I'm ready to agree that the lambda has internal linkage (being anonymous it's not accessible elsewhere in the same translation unit, so sure enough it's inaccessible in another one), but what about it. I don't want to access it from another translation unit.
The part about lacking definition strikes me as funny, I wouldn't even know how to write a lambda without defining it.
In summary: What gives? What to make of that? I don't like warnings, not only do they make the build less pretty, but they usually mean something is wrong and undefined behavior of sorts may bite you soon. On the other hand, how can I make a lambda more defined than it already is by writing out its definition?
It seems to me this is a compiler glitch. Using Clang compiler of Visual Studio 2017 only this error is generated "cannot refer to class template 'foo' without a template argument list" for a and b instantiation in main function. If the function type is specified as template parameter, there are no warnings and no errors.
Recently I ran into a few nonsense warnings that looked, and smelled, similar to yours. I tried a bunch of hand-wavey fixes with the idea that I could force the linker’s hand (if you will) – but at the end of the day I realized that I was trying to compile C++14 code with a -std=c++17 flag. You may want to double-check your own standards flags (and possibly your corresponding -stdlib=… flags as well).
Related
My understanding of function templates has always been: if they contain invalid
C++ and you don't instanciate them, your project will compile fine.
However, the following code:
#include <cstdio>
#include <utility>
template <typename T>
void contains_compile_time_error(T&& t) {
int j = nullptr;
}
int main() {}
Compiles with:
x86-64 gcc 11.2 and flag -std=c++20;
x64 msvc v19.31 and flag /std:c++20;
and does not with x86-64 clang 14.0.0 (with flags -std=c++{11,14,17,20}):
error: cannot initialize a variable of type 'int' with an rvalue of type 'std::nullptr_t'
int j = nullptr;
^ ~~~~~~~
1 error generated.
Even more confusing to me, the following code:
#include <cstdio>
#include <utility>
namespace ns {
struct S {};
template <typename T>
void apply(T&& t) {
//print(std::forward<T>(t));
ns::print(std::forward<T>(t));
}
void print(S const&) { std::puts("S"); }
} // namespace ns
int main() {}
Does not compile with:
x86-64 gcc 11.2 and flag -std=c++20;
x64 msvc v19.31 and flag /std:c++20;
x86-64 clang 14.0.0 and flags -std=c++{11,14,17,20};
(all of them complaining that print is not a member of 'ns') but does compile with x64 msvc v19.31 /std:c++17.
If I call unqualified print, then the code compiles with all the above compilers.
So, my questions are:
is my understanding of function templates wrong?
why do the above compilers behave differently with the code snippets I posted?
Edit 0: as per Frank's comment, x64 msvc v19.31 /std:c++17 /permissive- fails to compile function template apply where I call the qualified ns::print.
My understanding of function templates has always been: if they contain invalid C++ and you don't instanciate them, your project will compile fine.
Nope, if the code in the template is invalid, then so is the program.
But what does "invalid C++" mean? You can have syntactically valid C++ that is still invalid semantically, and the semantics of C++ are highly contextual.
So there's multiple levels of "valid" that can be checked at different times during compilation, based on the available information. Critically, there are things that can theoretically be checked early, but are potentially unreasonably difficult. Because of this, compilers are allowed some leeway when it comes to validating the semantics of template definitions. However, whether the code is broken or is not ambiguous, it's a compiler's ability to detect broken code that is.
why do the above compilers behave differently with the code snippets I posted?
As far as int j = nullptr is concerned:
The program is technically ill-formed, but the diagnostics is optional. So The code is broken, but GCC is not breaking compliance by letting it through.
For example, if S only had private constructors, then print(std::forward<T>(t)) would be flaggable as being broken since there is no possibly T that would make it valid. However, requiring compilers to be smart enough to determine this in all cases would be kind of mean.
For print():
Lookups are a bit of a different matter, there are hard rules that are supposed to be followed.
There are three types of lookups involved here.
Qualified lookups, such as ns::print
Unqualified lookups not involving template parameters, which you'd get if you tried print(S{});
Unqualified lookups involving template parameters, such as print(std::forward<T>(t));
Lookups are deferred for the third category, but must still be performed the same as non-template functions for the other two.
Note that the code still needs to be syntactically valid, even in the case of deferred lookups, hence why typename needs to be added when doing a dependent lookup of a type.
And as for MSVC allowing ns::print: That compiler is not compliant by default on this (and other) aspect of the standard. You need to use the /permissive- compiler flag to enforce compliance.
For a few days I've been using and testing my application without any trouble using the following code:
class dataHandler
{
public:
template<class T>
T GetData(bool truncate = false) { static_assert(false, "Unhandled output type"); }
template<T>
int GetData<int>(bool truncate)
{
// Normal stuff that compiles
}
}
This (As I expected at the time) works fine as long as no implicit instantiation of GetData's default is performed. However, today after adding a new failing specialisation for void* specifically (I wanted a different error) I discovered it wouldn't compile due to the assertion, even though the `void* spec was never called or mentioned in code.
I started a new C++ test project (with the same C++ version, MSVC version, and VS2022 version) and found this also doesn't compile for the same reason:
template<class T>
T Spec()
{
static_assert(false, "default");
}
template<>
int Spec<int>()
{
return 1;
}
I was unable to replicate the original 'success' in anything I tried within the test project, and the failing GetData<void*> assert in the initial project seems to indicate that it's not a project config / difference in toolset issue.
After some searching I discovered that it failing was the intended (or otherwise expected) behaviour, as well as a workaround for it.
However I find myself wondering why the initial case of static_assert(false) didn't also fail to compile. Is this some niche exception, or is this an issue with MSVC's consistency?
However I find myself wondering why the initial case of static_assert(false) didn't also fail to compile. Is this some niche exception, or is this an issue with MSVC's consistency?
It's certainly not an inconsistency. The key part here is "Standard conformance mode", controllable with the compiler option /permissive-. See the documentation of this compiler option.
The reason that MSVC from before VS2017 used to accept your static_assert(false, ...), is because it postponed parsing the contents of a function template until template instantiation. That is just how the parser used to work, and also why features like two-phase name lookup could never be properly implemented. I'll refer you to this blog post for more background on this.
You can easily try it for yourself. Even if the function contains ill-formed code that shouldn't even parse correctly, the compiler doesn't complain as long as the template isn't being instantiated:
template<class T>
void foo()
{
Normally this should not (never actually (compile)) but it does anyway
}
It appears to do some basic paranthesis matching and that is it. The token stream is recorded and parsed when foo is instantiated (usually when called).
In order to fix two-phase lookup (and other conformance issues), they had to fix the compiler and parse the code whenever a function template is encountered, rather than being instantiated. But now they have a problem, because a lot of old code might be reliant on the old compiler behavior that suddenly doesn't compile anymore. So they introduced the /permissive- option for users to opt-in into standard conformance mode. And gradually the default was changed for new projects or certain compiler options, as can be read in the documentation:
The /permissive- option is implicitly set by the /std:c++latest option starting in Visual Studio 2019 version 16.8, and in version 16.11 by the /std:c++20 option. /permissive- is required for C++20 Modules support. Perhaps your code doesn't need modules support but requires other features enabled under /std:c++20 or /std:c++latest. You can explicitly enable Microsoft extension support by using the /permissive option without the trailing dash. The /permissive option must come after any option that sets /permissive- implicitly.
By default, the /permissive- option is set in new projects created by Visual Studio 2017 version 15.5 and later versions. It's not set by default in earlier versions. When the option is set, the compiler generates diagnostic errors or warnings when non-standard language constructs are detected in your code. These constructs include some common bugs in pre-C++11 code.
Which brings us to the answer to your question. You weren't seeing it in your original code because you were compiling without /permissive-. In your test project, created in VS2022, /permissive- mode was set by default so it failed to compile. It also fails to compile in an explicit specialization (your void* case) because at that point the template arguments are known and the function is instantiated.
There are a couple of ways to properly fix your code. One is by explicitely deleting the main template and any specialiation you don't want to have.
template<class T> void foo() = delete;
template<> void foo<void*>() = delete;
template<> void foo<int>()
{
// ...
}
This will make any use of the explicitely deleted variants ill-formed, without having the option to include an error message. Of course the void* case is a bit redundant in this example.
If you want to stick to static_assert, you must make the condition for the assert dependent on the template arguments. If you are compiling with C++17, you could do:
template<class...> constexpr bool always_false = false;
template<class T> void foo()
{
static_assert(always_false<T>, "your error here");
}
If using C++14 or earlier, you could wrap the always_false into a struct template:
#include <type_traits>
template<class...> struct always_false : false_type { };
template<class T> void foo()
{
static_assert(always_false<T>::value, "your error here");
}
Reading questions, comments and answers on SO, I hear all the time that MSVC doesn't implement two-phase template lookup / instantiation correctly.
From what I understand so far, MSVC++ is only doing a basic syntax check on template classes and functions and doesn't check that names used in the template have atleast been declared or something along those lines.
Is this correct? What am I missing?
I'll just copy an example from my "notebook"
int foo(void*);
template<typename T> struct S {
S() { int i = foo(0); }
// A standard-compliant compiler is supposed to
// resolve the 'foo(0)' call here (i.e. early) and
// bind it to 'foo(void*)'
};
void foo(int);
int main() {
S<int> s;
// VS2005 will resolve the 'foo(0)' call here (i.e.
// late, during instantiation of 'S::S()') and
// bind it to 'foo(int)', reporting an error in the
// initialization of 'i'
}
The above code is supposed to compile in a standard C++ compiler. However, MSVC (2005 as well as 2010 Express) will report an error because of incorrect implementation of two-phase lookup.
And if you look closer, the issue is actually two-layered. At the surface, it is the obvious fact that Microsoft's compiler fails to perform early (first phase) lookup for a non-dependent expression foo(0). But what it does after that does not really behave as a proper implementation of the second lookup phase.
The language specification clearly states that during the second lookup phase only ADL-nominated namespaces get extended with additional declarations accumulated between the point of definition and point of instantiation. Meanwhile, non-ADL lookup (i.e. ordinary unqualified name lookup) is not extended by the second phase - it still sees those and only those declarations that were visible at the first phase.
That means that in the above example the compiler is not supposed to see void foo(int) at the second phase either. In other words, the MSVC's behavior cannot be described by a mere "MSVC postpones all lookup till the second phase". What MSVC implements is not a proper implementation of the second phase either.
To better illustrate the issue, consider the following example
namespace N {
struct S {};
}
void bar(void *) {}
template <typename T> void foo(T *t) {
bar(t);
}
void bar(N::S *s) {}
int main() {
N::S s;
foo(&s);
}
Note that even though bar(t) call inside the template definition is a dependent expression resolved at the second lookup phase, it should still resolve to void bar(void *). In this case ADL does not help the compiler to find void bar(N::S *s), while the regular unqualified lookup is not supposed to get "extended" by the second phase and thus is not supposed to see void bar(N::S *s) either.
Yet, Microsoft's compiler resolves the call to void bar(N::S *s). This is incorrect.
The problem is still present in its original glory in VS2015.
The Clang project has a pretty good writeup of two-phase lookup, and what the various implementation differences are: http://blog.llvm.org/2009/12/dreaded-two-phase-name-lookup.html
Short version: Two phase lookup is the name for the C++ standard defined behavior for name lookup within template code. Basically, some names are defined as dependent (the rules for which are a bit confusing), these names must be looked up when instantiating the template, and independent names must be looked up when parsing the template. This is both hard to implement (apparently), and confusing for developers, so compilers tend to not implement it to the standard. To answer your question, it looks like Visual C++ delays all lookups, but searches both the template context and the instantiation context, so it accepts a lot of code that the standard says it shouldn't. I'm not sure whether it doesn't accept code it should, or worse, interprets it differently, but it seems possible.
Historically gcc didn't implement the two-phase name lookup correctly either. It's apparently very difficult to get to, or at least there wasn't much incentive...
gcc 4.7 claims to implement it correctly, at last
CLang aims at implementing it, baring bugs, it's done on ToT and will get into 3.0
I don't know why VC++ writers never chose to implement this correctly, implementation of a similar behavior on CLang (for microsoft compabitility) hints that there might be some performance gain to delaying the instantiation of templates at the end of the translation unit (which does not mean implementing the look-up incorrectly, but make it even more difficult). Also, given the apparent difficulty of a correct implementation it may have been simpler (and cheaper).
I would note that VC++ is first, and foremost, a commercial product. It is driven by the need to satisfy its clients.
short answer
Disable language extensions with /Za
longer answer
I was investigating this issue lately and was amazed that under VS 2013 following example from standard [temp.dep]p3 produces wrong result:
typedef double A;
template<class T> class B {
public:
typedef int A;
};
template<class T> struct X : B<T> {
public:
A a;
};
int main()
{
X<int> x;
std::cout << "type of a: " << typeid(x.a).name() << std::endl;
}
will print:
type of a: int
while it should print double. The solution to make VS standard conformant is to disable language extensions (option /Za), now type of x.a will resolve to double, also other cases of using dependent names from base classes will be standard conformant. I am not sure if this enables two phase lookup.
[Update Jul-2019] Its also true for vs 2015 - https://rextester.com/YOH81784, but VS2019 shows correctly double. According to this article Two-phase name lookup support comes to MSVC, it was fixed since VS 2017.
Now that MSVC has most of two-phase name lookup implemented, I'm hoping this blog post answers this question completely: Two-phase name lookup comes to MSVC (VC++ blog)
I've been playing with the return type deduction supported in g++ with -std=c++1y.
If you prototype a function with an explicit return type, and then later try to define the function with return type deduction, the compiler complains of an ambiguous old declaration:
std::string some_function();
...
auto some_function(){ return std::string{"FOO"}; } //fails to compile
Is there a good reason why this doesn't work?
My rationale for using return type deduction in the definition is to keep the code clean, but want an explicit type in the prototype for self-documenting reasons. Recommendations on best practices for when and when not to use return type deduction would be appreciated :)
To be more clear, I would like answers to:
1. Is this an implementation mistake in the compiler? (I am fairly sure it is not)
2. Could this type of deduction be done, but isn't allowed by the proposal to the standard? If so, why not?
3. If this is really ambiguous, what are some examples where deducing the type and trying to match it with an explicit forward declaration would get you into trouble?
4. Are there deeper implementation specific issues behind this?
5. Is it simply an oversight?
It's because of how the function tables and overloaded functions work in C++.
When the compiler reads your std::string some_function(); it creates a spot for it to reference in the binary and says if "this function is ever called jump to this spot".
So we have a vtable that looks like this...
(Address offset) (Symbol)
0x???????? std::string somefunction();
Now it gets to your auto some_function() {...}. Normally it would first look in the function table to see if auto somefunction(); exists in the table or some variation there of, but the compiler notices that this is a implementation and it has the auto keyword so to reduce the entropy it writes *blank* some_function(); to the function table and ties to solve the return type.
Now the Function table looks like this...
(Address offset) (Symbol)
0x???????? std::string somefunction();
0x???????? ????? somefunction();
So it chugs along compiling code into binary when it finds out the return type which in this case is std::string. The compiler now knows what the return type is so it goes to the function table and changes auto somefunction(); too std::string somefunction();.
Now the Function table looks like this...
(Address offset) (Symbol)
0x???????? std::string somefunction();
0x???????? std::string somefunction();
Now the compiler goes back and continues to compile the function. Once it's done it goes back and finishes up the vtable only to find the same symbol is in there twice. It's now ambiguous to which symbol we are referring too.
So what is the reason for this?
Not 100% sure but vtables are made long before your code is worked down enough to allow for the deduction type to be made. So the only option the compiler has to work with at that stage in time is to just assume it's a new symbol. It's just something I've noticed looking at symbol tables all day and writing my own C++11 compiler.
I however can in no way speak for other compilers where lots of optimizations and other steps are introduced, I just work with the bare bones, however that is my understanding of the standard.
One last thing the type is completely arbitrary. It doesn't even matter if they don't match at the time of deduction. It never get's that far. The problem arises that there are two of the same symbols in the table, and you can't overload on return types.
you need to do it as:
auto some_function() -> decltype(std::string) { return std::string{"FOO"}; }
for more info look in http://en.wikipedia.org/wiki/C%2B%2B11 -> alternative function syntax
Reading questions, comments and answers on SO, I hear all the time that MSVC doesn't implement two-phase template lookup / instantiation correctly.
From what I understand so far, MSVC++ is only doing a basic syntax check on template classes and functions and doesn't check that names used in the template have atleast been declared or something along those lines.
Is this correct? What am I missing?
I'll just copy an example from my "notebook"
int foo(void*);
template<typename T> struct S {
S() { int i = foo(0); }
// A standard-compliant compiler is supposed to
// resolve the 'foo(0)' call here (i.e. early) and
// bind it to 'foo(void*)'
};
void foo(int);
int main() {
S<int> s;
// VS2005 will resolve the 'foo(0)' call here (i.e.
// late, during instantiation of 'S::S()') and
// bind it to 'foo(int)', reporting an error in the
// initialization of 'i'
}
The above code is supposed to compile in a standard C++ compiler. However, MSVC (2005 as well as 2010 Express) will report an error because of incorrect implementation of two-phase lookup.
And if you look closer, the issue is actually two-layered. At the surface, it is the obvious fact that Microsoft's compiler fails to perform early (first phase) lookup for a non-dependent expression foo(0). But what it does after that does not really behave as a proper implementation of the second lookup phase.
The language specification clearly states that during the second lookup phase only ADL-nominated namespaces get extended with additional declarations accumulated between the point of definition and point of instantiation. Meanwhile, non-ADL lookup (i.e. ordinary unqualified name lookup) is not extended by the second phase - it still sees those and only those declarations that were visible at the first phase.
That means that in the above example the compiler is not supposed to see void foo(int) at the second phase either. In other words, the MSVC's behavior cannot be described by a mere "MSVC postpones all lookup till the second phase". What MSVC implements is not a proper implementation of the second phase either.
To better illustrate the issue, consider the following example
namespace N {
struct S {};
}
void bar(void *) {}
template <typename T> void foo(T *t) {
bar(t);
}
void bar(N::S *s) {}
int main() {
N::S s;
foo(&s);
}
Note that even though bar(t) call inside the template definition is a dependent expression resolved at the second lookup phase, it should still resolve to void bar(void *). In this case ADL does not help the compiler to find void bar(N::S *s), while the regular unqualified lookup is not supposed to get "extended" by the second phase and thus is not supposed to see void bar(N::S *s) either.
Yet, Microsoft's compiler resolves the call to void bar(N::S *s). This is incorrect.
The problem is still present in its original glory in VS2015.
The Clang project has a pretty good writeup of two-phase lookup, and what the various implementation differences are: http://blog.llvm.org/2009/12/dreaded-two-phase-name-lookup.html
Short version: Two phase lookup is the name for the C++ standard defined behavior for name lookup within template code. Basically, some names are defined as dependent (the rules for which are a bit confusing), these names must be looked up when instantiating the template, and independent names must be looked up when parsing the template. This is both hard to implement (apparently), and confusing for developers, so compilers tend to not implement it to the standard. To answer your question, it looks like Visual C++ delays all lookups, but searches both the template context and the instantiation context, so it accepts a lot of code that the standard says it shouldn't. I'm not sure whether it doesn't accept code it should, or worse, interprets it differently, but it seems possible.
Historically gcc didn't implement the two-phase name lookup correctly either. It's apparently very difficult to get to, or at least there wasn't much incentive...
gcc 4.7 claims to implement it correctly, at last
CLang aims at implementing it, baring bugs, it's done on ToT and will get into 3.0
I don't know why VC++ writers never chose to implement this correctly, implementation of a similar behavior on CLang (for microsoft compabitility) hints that there might be some performance gain to delaying the instantiation of templates at the end of the translation unit (which does not mean implementing the look-up incorrectly, but make it even more difficult). Also, given the apparent difficulty of a correct implementation it may have been simpler (and cheaper).
I would note that VC++ is first, and foremost, a commercial product. It is driven by the need to satisfy its clients.
short answer
Disable language extensions with /Za
longer answer
I was investigating this issue lately and was amazed that under VS 2013 following example from standard [temp.dep]p3 produces wrong result:
typedef double A;
template<class T> class B {
public:
typedef int A;
};
template<class T> struct X : B<T> {
public:
A a;
};
int main()
{
X<int> x;
std::cout << "type of a: " << typeid(x.a).name() << std::endl;
}
will print:
type of a: int
while it should print double. The solution to make VS standard conformant is to disable language extensions (option /Za), now type of x.a will resolve to double, also other cases of using dependent names from base classes will be standard conformant. I am not sure if this enables two phase lookup.
[Update Jul-2019] Its also true for vs 2015 - https://rextester.com/YOH81784, but VS2019 shows correctly double. According to this article Two-phase name lookup support comes to MSVC, it was fixed since VS 2017.
Now that MSVC has most of two-phase name lookup implemented, I'm hoping this blog post answers this question completely: Two-phase name lookup comes to MSVC (VC++ blog)