I am implementing a sorting algorithm as personal training (no homework!).
I have the following code (excluding imports etc.):
template<class RandomIt, class Compare>
void sort(RandomIt first, RandomIt last, Compare comp)
{
/* actual sorting code is here */
}
template<class RandomIt>
void sort(RandomIt first, RandomIt last)
{
std::function<bool(decltype(*first), decltype(*last))> comp = [](decltype(*first) a, decltype(*last) b)
{
return a < b;
};
sort (first, last, comp);
}
Trying to call this code with a test array
auto test_array_1 = std::make_unique <std::array < uint64_t,SORTING_TEST_LENGTH >> ();
std::copy(std::cbegin(*template_array), std::cend(*template_array), std::begin(*test_array_1));
sort(std::begin(*test_array_1), std::end(*test_array_1));
The compiler complains about "ambiguous call to overloaded function" (VC++ C2668). From my understanding the call should not be ambiguous though. Also giving the call in the second sort function the template parameters for the first sort function does not have any effect.
What am I missing here? Why does the compiler consider the second call "ambiguous"?
The problem is two fold.
First, sort is found via ADL, so you get two overloads, and they both match. In general, naming functions the same as std functions when you are not trying to ADL-overload is fraught, due to the possibility of ADL-induced ambiguity.
Now, this only happens when passing types from namespace std; Sometimes iterators are from this namespace, but not in this case : array uses raw pointer iterators. The trigger for ADL finding std::sort is the std::function.
Which brings me to the next problem: there is very little to be gained, and much to be lost, by std::function in the above code. Replace it with auto. Pass low level sorting algorithms an inlineable comparison object.
You still do not want to call it sort. If you call it sort you need to qualify calls with the namespace, or (sort) to block ADL.
The ADL rule is that both the "usual" functions, and functions in the namespace of arguments, and namespace of what arguments point to, and template arguments of arguments, etc, are considered for overload resolution. This is Argument Dependent Lookup, or ADL, or Koenig lookup. It means that namespace pollution of a kind can occur when using types from another namespace (which is sad), but it also makes some nice magic happen (like std::cout << "hello world\n";`)
Related
I am implementing a sorting algorithm as personal training (no homework!).
I have the following code (excluding imports etc.):
template<class RandomIt, class Compare>
void sort(RandomIt first, RandomIt last, Compare comp)
{
/* actual sorting code is here */
}
template<class RandomIt>
void sort(RandomIt first, RandomIt last)
{
std::function<bool(decltype(*first), decltype(*last))> comp = [](decltype(*first) a, decltype(*last) b)
{
return a < b;
};
sort (first, last, comp);
}
Trying to call this code with a test array
auto test_array_1 = std::make_unique <std::array < uint64_t,SORTING_TEST_LENGTH >> ();
std::copy(std::cbegin(*template_array), std::cend(*template_array), std::begin(*test_array_1));
sort(std::begin(*test_array_1), std::end(*test_array_1));
The compiler complains about "ambiguous call to overloaded function" (VC++ C2668). From my understanding the call should not be ambiguous though. Also giving the call in the second sort function the template parameters for the first sort function does not have any effect.
What am I missing here? Why does the compiler consider the second call "ambiguous"?
The problem is two fold.
First, sort is found via ADL, so you get two overloads, and they both match. In general, naming functions the same as std functions when you are not trying to ADL-overload is fraught, due to the possibility of ADL-induced ambiguity.
Now, this only happens when passing types from namespace std; Sometimes iterators are from this namespace, but not in this case : array uses raw pointer iterators. The trigger for ADL finding std::sort is the std::function.
Which brings me to the next problem: there is very little to be gained, and much to be lost, by std::function in the above code. Replace it with auto. Pass low level sorting algorithms an inlineable comparison object.
You still do not want to call it sort. If you call it sort you need to qualify calls with the namespace, or (sort) to block ADL.
The ADL rule is that both the "usual" functions, and functions in the namespace of arguments, and namespace of what arguments point to, and template arguments of arguments, etc, are considered for overload resolution. This is Argument Dependent Lookup, or ADL, or Koenig lookup. It means that namespace pollution of a kind can occur when using types from another namespace (which is sad), but it also makes some nice magic happen (like std::cout << "hello world\n";`)
I am practicing C++ by building my own version of vector, named "Vector". I have two constructors among others, fill constructor and range constructor. Their declarations look like following:
template <typename Type>
class Vector {
public:
// fill constructor
Vector(size_t num, const Type& cont);
// range constructor
template<typename InputIterator> Vector(InputIterator first, InputIterator last);
/*
other members
......
*/
}
The fill constructor fills the container with num of val; and the range constructor copies every value in the range [first, last) into the container. They are supposed to be the same with the two constructors of the STL vector.
Their definitions goes following:
//fill constructor
template <typename Type>
Vector<Type>::Vector(size_t num, const Type& cont){
content = new Type[num];
for (int i = 0; i < num; i ++)
content[i] = cont;
contentSize = contentCapacity = num;
}
// range constructor
template <typename Type>
template<typename InputIterator>
Vector<Type>::Vector(InputIterator first, InputIterator last){
this->content = new Type[last - first];
int i = 0;
for (InputIterator iitr = first; iitr != last; iitr ++, i ++)
*(content + i) = *iitr;
this->contentSize = this->contentCapacity = i;
}
However, when I try to use them, I have problem distinguishing them.
For example:
Vector<int> v1(3, 5);
With the this line of code, I intended to create a Vector that contains three elements, each of which is 5. But the compiler goes for the range constructor, treating both "3" and "5" as instances of the "InputIterator", which, with no surprises, causes error.
Of course, if I change the code to:
Vector<int> v1(size_t(3), 5);
Everything is fine, the fill constructor is called. But that is obviously not intuitive and user friendly.
So, is there a way that I can use the fill constructor intuitively?
You can use std::enable_if (or boost::enable_if if you don't use C++11) to disambiguate the constructors.
#include <iostream>
#include <type_traits>
#include <vector>
using namespace std;
template <typename Type>
class Vector {
public:
// fill constructor
Vector(size_t num, const Type& cont)
{
cout << "Fill constructor" << endl;
}
// range constructor
template<typename InputIterator> Vector(InputIterator first, InputIterator last,
typename std::enable_if<!std::is_integral<InputIterator>::value>::type* = 0)
{
cout << "Range constructor" << endl;
}
};
int main()
{
Vector<int> v1(3, 5);
std::vector<int> v2(3, 5);
Vector<int> v3(v2.begin(), v2.end());
}
The above program should first call the fill constructor by checking if the type is an integral type (and thus not an iterator.)
By the way, in your implementation of the range constructor, you should use std::distance(first, last) rather than last - first. Explicitly using the - operator on iterators limits you to RandomAccessIterator types, but you want to support InputIterator which is the most generic type of Iterator.
Even std::vector seems to have this issue.
std::vector<int> v2(2,3);
chooses
template<class _Iter>
vector(_Iter _First, _Iter _Last)
In Visual C++, even though it should match closer to the non templated case..
Edit: That above function (correctly) delegates the construction to the below one. I am totally lost..
template<class _Iter>
void _Construct(_Iter _Count, _Iter _Val, _Int_iterator_tag)
Edit #2 AH!:
Somehow this below function identifies which version of the constructor is meant to be called.
template<class _Iter> inline
typename iterator_traits<_Iter>::iterator_category
_Iter_cat(const _Iter&)
{ // return category from iterator argument
typename iterator_traits<_Iter>::iterator_category _Cat;
return (_Cat);
}
The above shown _Construct function has (atleast) 2 versions overloading on the third variable which is a tag to returned by the above _Iter_cat function. Based on the type of this category the correct overload of the _Construct is picked.
Final edit:
iterator_traits<_Iter> is a class that seems to be templated for many different common varieties, each returning the appropriate "Category" type
Solution: It appears template specialization of the first arguement's type is how the std library handles this messy situation (primitive value type) in the case of MS VC++. Perhaps you could look into it and follow suit?
The problem arises (I think) because with primitive value types, the Type and size_t variables are similar, and so the template version with two identical types gets picked.
The problem is the same at the one faced by the standard library implementation. There are several ways to solve it.
You can meticulously provide non-template overloaded constructors for all integral types (in place of the first parameter).
You can use SFINAE-based technique (like enable_if) to make sure the range constructor is not selected for integer argument.
You can branch the range constructor at run-time (by using if) after detecting integral argument (by using is_integral) and redirect control to the proper constructing code. The branching condition will be a compile-time value, meaning that the code will probably be reduced at compile-time by the compiler.
You can simply peek into your version of standard library implementation and see how they do it (although their approach is not required to be portable and/or valid from the point of view of abstract C++ language).
This ambiguity caused problems for early library implementers. It's called the "Do The Right Thing" effect. As far as I know, you need SFINAE to solve it… it might have been one of the first applications of that technique. (Some compilers cheated and hacked their overload resolution internals, until the solution was found within the core language.)
The standard specification of this issue is one of the key differences between C++98 and C++03. From C++11, §23.2.3:
14 For every sequence container defined in this Clause and in Clause 21:
— If the constructor
template <class InputIterator>
X(InputIterator first, InputIterator last,
const allocator_type& alloc = allocator_type())
is called with a type InputIterator that does not qualify as an input iterator, then the constructor shall not participate in overload resolution.
15 The extent to which an implementation determines that a type cannot be an input iterator is unspecified, except that as a minimum integral types shall not qualify as input iterators.
Inspired by contains(), I want do declare contains(), fitting misc containers.
// for std::vector, std::list, .. (1)
template<typename C, typename T>
bool contains(const C& container, const T& val)
{
return ::std::find(container.begin(), container.end(), val) != container.end();
}
// partial specialization for std::map (2)
template<typename K, typename V>
bool contains(const ::std::map<K, V>& container, const K& key)
{
// std::map.find() is better than std::find used in (1)
return container.find(key) != container.end();
}
Following question, I want to add the functions to namespace the arguments belongs too.
Questions:
Types of val from (1) and key from (2) are unknown. Does it mean I don't need to put the functions to any namespace or do I need to put them in std namespace, the containers belong to?
Is it possible to improve (2) it will preferred by compiler than (1) for std::map, std::set, boost::unordered_map, boost::unordered_set?
boost::algorithm::contains(). Do I need choose another name for my wrapper?
(1) Don't add anything to the std namespace
(2) As Space_C0wb0y points out, (2) is not a partial specialization, but an overload. I'm not sure if this is standard behaviour, but on VS2008 this is resolved correctly (the overloaded version is used for map). In either case, I think a slighly better version would be:
template<typename C>
bool contains(const C & mapContainer, const typename C::key_type & key)
{
// std::map.find() is better than std::find used in (1)
return mapContainer.find(key) != mapContainer.end();
}
This is a partial specialization (T is specialized to C::key_type) and would work for all types that has the key_type typedef (e.g. std::map, boost::unordered_map etc.)
(3) Put in separate namespace (see #Tony's answer)
Sidenote: I'm not sure I think these functions should have the same name. If I saw a function contains() accepting a map + another argument I could think that (1) the function checks if the map contains an entry for the given value (e.g. some key has the value provided) or (2) the function checks if there is such an entry in the map (e.g. the provided value is a pair). Instead I would call the function contains_key() for maps.
Edit:
After a bit of checking I suspect that the code above isn't compliant as it uses "template typedefs". This is not a standard C++ feature, but it seems that Visual Studio supports it in some cases. There's probably a workaround for this, but not one I'm clever enough to come up with now.
On number 3. you should place your version of contains in it's own namespace to avoid naming clashes. That's the purpose of namespaces.
Yes, keep them in a separate namespace.
The only thing to place in the same namespace as the argument type is operator overloads, to allow for Koenig (argument-dependent) lookup. These are not overloaded operators. End of story.
Note how this also answer question 2.: how you make the compiler prefer your implementations:
using myownversions::contains;
(assuming you named your namespace myownversions for now)
You must not put them into the std namespace. This is forbidden. You can put them into any namespace you like.
There is probably some SFINAE-magic you can do there, but I cannot come up with it in my own.
See #Tony's answer.
I've been bitten by this problem a couple of times and so have my colleagues. When compiling
#include <deque>
#include <boost/algorithm/string/find.hpp>
#include <boost/operators.hpp>
template< class Rng, class T >
typename boost::range_iterator<Rng>::type find( Rng& rng, T const& t ) {
return std::find( boost::begin(rng), boost::end(rng), t );
}
struct STest {
bool operator==(STest const& test) const { return true; }
};
struct STest2 : boost::equality_comparable<STest2> {
bool operator==(STest2 const& test) const { return true; }
};
void main() {
std::deque<STest> deq;
find( deq, STest() ); // works
find( deq, STest2() ); // C2668: 'find' : ambiguous call to overloaded function
}
...the VS9 compiler fails when compiling the second find. This is due to the fact that STest2 inherits from a type that is defined in boost namespace which triggers the compiler to try ADL which finds boost::algorithm::find(RangeT& Input, const FinderT& Finder).
An obvious solution is to prefix the call to find(…) with "::" but why is this necessary? There is a perfectly valid match in the global namespace, so why invoke Argument-Dependent Lookup? Can anybody explain the rationale here?
ADL isn't a fallback mechanism to use when "normal" overload resolution fails, functions found by ADL are just as viable as functions found by normal lookup.
If ADL was a fallback solution then you might easily fall into the trap were a function was used even when there was another function that was a better match but only visible via ADL. This would seem especially strange in the case of (for example) operator overloads. You wouldn't want two objects to be compared via an operator== for types that they could be implicitly converted to when there exists a perfectly good operator== in the appropriate namespace.
I'll add the obvious answer myself because I just did some research on this problem:
C++03 3.4.2
§2 For each argument type T in the function call, there is a set of zero or more associated namespaces [...] The sets of namespaces and classes are determined in the following way:
[...]
— If T is a class type (including unions), its associated classes are: the class itself; the class of which it is a
member, if any; and its direct and indirect base classes. Its associated namespaces are the namespaces
in which its associated classes are defined.
§ 2a If the ordinary unqualified lookup of the name finds the declaration of a class member function, the associated
namespaces and classes are not considered. Otherwise the set of declarations found by the lookup of
the function name is the union of the set of declarations found using ordinary unqualified lookup and the set
of declarations found in the namespaces and classes associated with the argument types.
At least it's standard conformant, but I still don't understand the rationale here.
Consider a mystream which inherits from std::ostream. You would like that your type would support all the << operators that are defined for std::ostream normally in the std namespace. So base classes are associated classes for ADL.
I think this also follows from the substitution principle - and functions in a class' namespace are considered part of its interface (see Herb Sutter's "What's in a class?"). So an interface that works on the base class should remain working on a derived class.
You can also work around this by disabling ADL:
(find)( deq, STest2() );
I think you stated the problem yourself:
in the global namespace
Functions in the global namespace are considered last. It's the most outer scope by definition. Any function with the same name (not necessarily applicable) that is found in a closer scope (from the call point of view) will be picked up first.
template <typename Rng, typename T>
typename Rng::iterator find( Rng& rng, T const& t );
namespace foo
{
bool find(std::vector<int> const& v, int);
void method()
{
std::deque<std::string> deque;
auto it = find(deque, "bar");
}
}
Here (unless vector or deque include algorithm, which is allowed), the only method that will be picked up during name look-up will be:
bool foo::find(std::vector<int> const&, int);
If algorithm is somehow included, there will also be:
template <typename FwdIt>
FwdIt std::find(FwdIt begin, FwdIt end,
typename std::iterator_traits<FwdIt>::value_type const& value);
And of course, overload resolution will fail stating that there is no match.
Note that name-lookup is extremely dumb: neither arity nor argument type are considered!
Therefore, there are only two kinds of free-functions that you should use in C++:
Those which are part of the interface of a class, declared in the same namespace, picked up by ADL
Those which are not, and that you should explicitly qualified to avoid issues of this type
If you fall out of these rules, it might work, or not, depending on what's included, and that's very awkward.
Some time ago I read an article that explained several pitfalls of argument dependent lookup, but I cannot find it anymore. It was about gaining access to things that you should not have access to or something like that. So I thought I'd ask here: what are the pitfalls of ADL?
There is a huge problem with argument-dependent lookup. Consider, for example, the following utility:
#include <iostream>
namespace utility
{
template <typename T>
void print(T x)
{
std::cout << x << std::endl;
}
template <typename T>
void print_n(T x, unsigned n)
{
for (unsigned i = 0; i < n; ++i)
print(x);
}
}
It's simple enough, right? We can call print_n() and pass it any object and it will call print to print the object n times.
Actually, it turns out that if we only look at this code, we have absolutely no idea what function will be called by print_n. It might be the print function template given here, but it might not be. Why? Argument-dependent lookup.
As an example, let's say you have written a class to represent a unicorn. For some reason, you've also defined a function named print (what a coincidence!) that just causes the program to crash by writing to a dereferenced null pointer (who knows why you did this; that's not important):
namespace my_stuff
{
struct unicorn { /* unicorn stuff goes here */ };
std::ostream& operator<<(std::ostream& os, unicorn x) { return os; }
// Don't ever call this! It just crashes! I don't know why I wrote it!
void print(unicorn) { *(int*)0 = 42; }
}
Next, you write a little program that creates a unicorn and prints it four times:
int main()
{
my_stuff::unicorn x;
utility::print_n(x, 4);
}
You compile this program, run it, and... it crashes. "What?! No way," you say: "I just called print_n, which calls the print function to print the unicorn four times!" Yes, that's true, but it hasn't called the print function you expected it to call. It's called my_stuff::print.
Why is my_stuff::print selected? During name lookup, the compiler sees that the argument to the call to print is of type unicorn, which is a class type that is declared in the namespace my_stuff.
Because of argument-dependent lookup, the compiler includes this namespace in its search for candidate functions named print. It finds my_stuff::print, which is then selected as the best viable candidate during overload resolution: no conversion is required to call either of the candidate print functions and nontemplate functions are preferred to function templates, so the nontemplate function my_stuff::print is the best match.
(If you don't believe this, you can compile the code in this question as-is and see ADL in action.)
Yes, argument-dependent lookup is an important feature of C++. It is essentially required to achieve the desired behavior of some language features like overloaded operators (consider the streams library). That said, it's also very, very flawed and can lead to really ugly problems. There have been several proposals to fix argument-dependent lookup, but none of them have been accepted by the C++ standards committee.
The accepted answer is simply wrong - this is not a bug of ADL. It shows an careless anti-pattern to use function calls in daily coding - ignorance of dependent names and relying on unqualified function names blindly.
In short, if you are using unqualified name in the postfix-expression of a function call, you should have acknowledged that you have granted the ability that the function can be "overridden" elsewhere (yes, this is a kind of static polymorphism). Thus, the spelling of the unqualified name of a function in C++ is exactly a part of the interface.
In the case of the accepted answer, if the print_n really need ADL print (i.e. allowing it to be overridden), it should have been documented with the use of unqualified print as an explicit notice, thus clients would receive a contract that print should be carefully declared and the misbehavior would be all of the responsibility of my_stuff. Otherwise, it is a bug of print_n. The fix is simple: qualify print with prefix utility::. This is indeed a bug of print_n, but hardly a bug of the ADL rules in the language.
However, there do exist unwanted things in the language specification, and technically, not only one. They are realized more than 10 years, but nothing in the language is fixed yet. They are missed by the accepted answer (except that the last paragraph is solely correct till now). See this paper for details.
I can append one real case against the name lookup nasty. I was implementing is_nothrow_swappable where __cplusplus < 201703L. I found it impossible to rely on ADL to implementing such feature once I have a declared swap function template in my namespace. Such swap would always found together with std::swap introduced by a idiomatic using std::swap; to use ADL under the ADL rules, and then there would come ambiguity of swap where the swap template (which would instantiate is_nothrow_swappable to get the proper noexcept-specification) is called. Combined with 2-phase lookup rules, the order of declarations does not count, once the library header containing the swap template is included. So, unless I overload all my library types with specialized swap function (to supress any candidate generic templates swap being matched by overloading resolution after ADL), I cannot declare the template. Ironically, the swap template declared in my namespace is exactly to utilize ADL (consider boost::swap) and it is one of the most significant direct client of is_nothrow_swappable in my library (BTW, boost::swap does not respects the exception specification). This perfectly beat my purpose up, sigh...
#include <type_traits>
#include <utility>
#include <memory>
#include <iterator>
namespace my
{
#define USE_MY_SWAP_TEMPLATE true
#define HEY_I_HAVE_SWAP_IN_MY_LIBRARY_EVERYWHERE false
namespace details
{
using ::std::swap;
template<typename T>
struct is_nothrow_swappable
: std::integral_constant<bool, noexcept(swap(::std::declval<T&>(), ::std::declval<T&>()))>
{};
} // namespace details
using details::is_nothrow_swappable;
#if USE_MY_SWAP_TEMPLATE
template<typename T>
void
swap(T& x, T& y) noexcept(is_nothrow_swappable<T>::value)
{
// XXX: Nasty but clever hack?
std::iter_swap(std::addressof(x), std::addressof(y));
}
#endif
class C
{};
// Why I declared 'swap' above if I can accept to declare 'swap' for EVERY type in my library?
#if !USE_MY_SWAP_TEMPLATE || HEY_I_HAVE_SWAP_IN_MY_LIBRARY_EVERYWHERE
void
swap(C&, C&) noexcept
{}
#endif
} // namespace my
int
main()
{
my::C a, b;
#if USE_MY_SWAP_TEMPLATE
my::swap(a, b); // Even no ADL here...
#else
using std::swap; // This merely works, but repeating this EVERYWHERE is not attractive at all... and error-prone.
swap(a, b); // ADL rocks?
#endif
}
Try https://wandbox.org/permlink/4pcqdx0yYnhhrASi and turn USE_MY_SWAP_TEMPLATE to true to see the ambiguity.
Update 2018-11-05:
Aha, I am bitten by ADL this morning again. This time it even has nothing to do with function calls!
Today I am finishing the work of porting ISO C++17 std::polymorphic_allocator to my codebase. Since some container class templates have been introduced long ago in my code (like this), this time I just replace the declarations with alias templates like:
namespace pmr = ystdex::pmr;
template<typename _tKey, typename _tMapped, typename _fComp
= ystdex::less<_tKey>, class _tAlloc
= pmr::polymorphic_allocator<std::pair<const _tKey, _tMapped>>>
using multimap = std::multimap<_tKey, _tMapped, _fComp, _tAlloc>;
... so it can use my implementation of polymorphic_allocator by default. (Disclaimer: it has some known bugs. Fixes of the bugs would be committed in a few days.)
But it suddenly does not work, with hundreds of lines of cryptic error messages...
The error begins from this line. It roughly complains that the declared BaseType is not a base of the enclosing class MessageQueue. That seems very strange because the alias is declared with exactly the same tokens to those in the base-specifier-list of the class definition, and I am sure nothing of them can be macro-expanded. So why?
The answer is... ADL sucks. The line inroducing BaseType is hard-coded with a std name as a template argument, so the template would be looked up per ADL rules in the class scope. Thus, it finds std::multimap, which differs to the result of lookup in as the actual base class declared in the enclosing namespace scope. Since std::multimap uses std::allocator instance as the default template argument, BaseType is not the same type to the actual base class which have an instance of polymorphic_allocator, even multimap declared in the enclosing namespace is redirected to std::multimap. By adding the enclosing qualification as the prefix right to the =, the bug is fixed.
I'd admit I am lucky enough. The error messages are heading the problem to this line. There are only 2 similar problems and the other is without any explicit std (where string is my own one being adapted to ISO C++17's string_view change, not std one in pre-C++17 modes). I would not figure out the bug is about ADL so quickly.