Considering:
#include <cassert>
#include <boost/range/irange.hpp>
#include <boost/range/algorithm.hpp>
int main() {
auto range = boost::irange(1, 4);
assert(boost::find(range, 4) == end(range));
}
Live Clang demo
Live GCC demo
this gives:
main.cpp:8:37: error: use of undeclared identifier 'end'
Considering that if you write using boost::end; it works just fine, which implies that boost::end is visible:
Why is ADL not working and finding boost::end in the expression end(range)? And if it's intentional, what's the rationale behind it?
To be clear, the expected result would be similar to what happens in this example using std::find_if and unqualified end(vec).
Historical background
The underlying reason is discussed in this closed Boost ticket
With the following code, compiler will complain that no begin/end is
found for "range_2" which is integer range. I guess that integer range
is missing ADL compatibility ?
#include <vector>
#include <boost/range/iterator_range.hpp>
#include <boost/range/irange.hpp>
int main() {
std::vector<int> v;
auto range_1 = boost::make_iterator_range(v);
auto range_2 = boost::irange(0, 1);
begin(range_1); // found by ADL
end(range_1); // found by ADL
begin(range_2); // not found by ADL
end(range_2); // not found by ADL
return 0;
}
boost::begin() and boost::end() are not meant to be found by ADL. In
fact, Boost.Range specifically takes precautions to prevent
boost::begin() and boost::end() from being found by ADL, by declaring
them in the namespace boost::range_adl_barrier and then exporting them
into the namespace boost from there. (This technique is called an "ADL
barrier").
In the case of your range_1, the reason unqualified begin() and end()
calls work is because ADL looks not only at the namespace a template
was declared in, but the namespaces the template arguments were
declared in as well. In this case, the type of range_1 is
boost::iterator_range<std::vector<int>::iterator>. The template
argument is in namespace std (on most implementations), so ADL finds
std::begin() and std::end() (which, unlike boost::begin() and
boost::end(), do not use an ADL barrier to prevent being found by
ADL).
To get your code to compile, simply add "using boost::begin;" and
"using boost::end;", or explicitly qualify your begin()/end() calls
with "boost::".
Extended code example illustrating the dangers of ADL
The danger of ADL from unqualified calls to begin and end is two-fold:
the set of associated namespaces can be much larger than one expects. E.g. in begin(x), if x has (possibly defaulted!) template parameters, or hidden base classes in its implementation, the associated namespaces of the template parameters and of its base classes are also considered by ADL. Each of those associated namespace can lead to many overloads of begin and end being pulled in during argument dependent lookup.
unconstrained templates cannot be distinguished during overload resolution. E.g. in namespace std, the begin and end function templates are not separately overloaded for each container, or otherwise constrained on the signature of the container being supplied. When another namespace (such as boost) also supplies similarly unconstrained function templates, overload resolution will consider both an equal match, and an error occurs.
The following code samples illustrate the above points.
A small container library
The first ingredient is to have a container class template, nicely wrapped in its own namespace, with an iterator that derives from std::iterator, and with generic and unconstrained function templates begin and end.
#include <iostream>
#include <iterator>
namespace C {
template<class T, int N>
struct Container
{
T data[N];
using value_type = T;
struct Iterator : public std::iterator<std::forward_iterator_tag, T>
{
T* value;
Iterator(T* v) : value{v} {}
operator T*() { return value; }
auto& operator++() { ++value; return *this; }
};
auto begin() { return Iterator{data}; }
auto end() { return Iterator{data+N}; }
};
template<class Cont>
auto begin(Cont& c) -> decltype(c.begin()) { return c.begin(); }
template<class Cont>
auto end(Cont& c) -> decltype(c.end()) { return c.end(); }
} // C
A small range library
The second ingredient is to have a range library, also wrapped in its own namespace, with another set of unconstrained function templates begin and end.
namespace R {
template<class It>
struct IteratorRange
{
It first, second;
auto begin() { return first; }
auto end() { return second; }
};
template<class It>
auto make_range(It first, It last)
-> IteratorRange<It>
{
return { first, last };
}
template<class Rng>
auto begin(Rng& rng) -> decltype(rng.begin()) { return rng.begin(); }
template<class Rng>
auto end(Rng& rng) -> decltype(rng.end()) { return rng.end(); }
} // R
Overload resolution ambiguity through ADL
Trouble begins when one tries to make an iterator range into a container, while iterating with unqualified begin and end:
int main()
{
C::Container<int, 4> arr = {{ 1, 2, 3, 4 }};
auto rng = R::make_range(arr.begin(), arr.end());
for (auto it = begin(rng), e = end(rng); it != e; ++it)
std::cout << *it;
}
Live Example
Argument-dependent name lookup on rng will find 3 overloads for both begin and end: from namespace R (because rng lives there), from namespace C (because the rng template parameter Container<int, 4>::Iterator lives there), and from namespace std (because the iterator is derived from std::iterator). Overload resolution will then consider all 3 overloads an equal match and this results in a hard error.
Boost solves this by putting boost::begin and boost::end in an inner namespace and pulling them into the enclosing boost namespace by using directives. An alternative, and IMO more direct way, would be to ADL-protect the types (not the functions), so in this case, the Container and IteratorRange class templates.
Live Example With ADL barriers
Protecting your own code may not be enough
Funny enough, ADL-protecting Container and IteratorRange would -in this particular case- be enough to let the above code run without error because std::begin and std::end would be called because std::iterator is not ADL-protected. This is very surprising and fragile. E.g. if the implementation of C::Container::Iterator no longer derives from std::iterator, the code would stop compiling. It is therefore preferable to use qualified calls R::begin and R::end on any range from namespace R in order to be protected from such underhanded name-hijacking.
Note also that the range-for used to have the above semantics (doing ADL with at least std as an associated namespace). This was discussed in N3257 which led to semantic changes in range-for. The current range-for first looks for member functions begin and end, so that std::begin and std::end will not be considered, regardless of ADL-barriers and inheritance from std::iterator.
int main()
{
C::Container<int, 4> arr = {{ 1, 2, 3, 4 }};
auto rng = R::make_range(arr.begin(), arr.end());
for (auto e : rng)
std::cout << e;
}
Live Example
In boost/range/end.hpp they explicitly block ADL by putting end in a range_adl_barrier namespace, then using namespace range_adl_barrier; to bring it into the boost namespace.
As end is not actually from ::boost, but rather from ::boost::range_adl_barrier, it is not found by ADL.
Their reasoning is described in boost/range/begin.hpp:
// Use a ADL namespace barrier to avoid ambiguity with other unqualified
// calls. This is particularly important with C++0x encouraging
// unqualified calls to begin/end.
no examples are given of where this causes a problem, so I can only theorize what they are talking about.
Here is an example I have invented of how ADL can cause ambiguity:
namespace foo {
template<class T>
void begin(T const&) {}
}
namespace bar {
template<class T>
void begin(T const&) {}
struct bar_type {};
}
int main() {
using foo::begin;
begin( bar::bar_type{} );
}
live example. Both foo::begin and bar::begin are equally valid functions to call for the begin( bar::bar_type{} ) in that context.
This could be what they are talking about. Their boost::begin and std::begin might be equally valid in a context where you have using std::begin on a type from boost. By putting it in a sub-namespace of boost, std::begin gets called (and works on ranges, naturally).
If the begin in the namespace boost had been less generic, it would be preferred, but that isn't how they wrote it.
That's because boost::end is inside an ADL barrier, which is then pulled in boost at the end of the file.
However, from cppreference's page on ADL (sorry, I don't have a C++ draft handy):
1) using-directives in the associated namespaces are ignored
That prevents it from being included in ADL.
Related
According to cppreference, in Argument-dependent lookups
function names are looked up in the namespaces of their arguments in
addition to the scopes and namespaces considered by the usual
unqualified name lookup.
However, why is it then that in the following example in function A::member_function() the class-method begin() is choosen which 1) takes 0 arguments and 2) isn't even in the namespace of std::vector, the argument of the function? Shouldn't ADL go to the namespace of the arguments (which is std for vector) and choose the free function std::begin()?
#include <unordered_map>
#include <vector>
class A
{
public:
using value_t = std::unordered_map<int, char>;
public:
void member_function();
value_t::iterator begin() { return map_.begin(); }
private:
value_t map_;
};
void A::member_function()
{
std::vector<int> hello;
auto it = begin(hello);
}
I'm also happy for a workaround if this is really the way ADL works.
ADL is not done if ordinary unqualified name lookup finds a class member.
You can avoid this issue with a using-declaration, so that unqualified name lookup is guaranteed to always result in std::begin:
using std::begin;
auto it = begin(hello);
This will always find std::begin, but also do ADL to find other begin overloads.
You can also use a qualified call std::begin(hello); which will forward correctly to the .begin member of the argument, if possible, but will not do any ADL.
In C++20 there is std::ranges::begin, which can be used instead of the method above. It is a customization point object and will forward to the correct begin for the argument more generally, either to a member if possible or via ADL.
Let's assume I have the following Data class:
struct Data {
char foo[8];
char bar;
};
and the following function, my_algorithm, which takes a pair of char * (similar to an STL algorithm):
void my_algorithm(char *first, char *last);
For Data's foo data member, instead of calling my_algorithm() like this:
Data data;
my_algorithm(data.foo, data.foo + 8);
I can use the std::begin() and std::end() convenience function templates:
my_algorithm(std::begin(data.foo), std::end(data.foo));
I would like to achieve something similar to Data's bar data member. That is, instead of writing:
my_algorithm(&data.bar, &data.bar + 1);
I would like to write something like:
my_algorithm(begin(data.bar), end(data.bar));
Therefore, I've defined the two following ordinary (non-template) functions for this case:
char* begin(char& c) { return &c; }
char* end(char& c) { return &c + 1; }
So that I would be able to write code like the following:
Data data;
using std::begin;
using std::end;
my_algorithm(begin(data.foo), end(data.foo)); // ok - std::begin()/std::end()
my_algorithm(begin(data.bar), end(data.bar)); // Error!!!
With the using declarations above I would have expected std::begin()/std::end() and ::begin()/::end() to be in the same overload set, respectively. Since the functions ::begin() and ::end() are a perfect match for the latter call and they are not templates, I would have expected the last call to my_algorithm() to match them. However, the ordinary functions are not considered at all. As a result the compilation fails, because std::begin() and std::end() are not matches for the call.
Basically, the latter call acts as if I had written instead:
my_algorithm(begin<>(data.bar), end<>(data.bar));
That is, only the function templates (i.e., std::begin()/std::end()) are considered by the overload resolution process, not the ordinary functions (i.e., not ::begin()/::end()).
It only works as expected, if I fully qualify the calls to ::begin()/::end():
my_algorithm(::begin(data.bar), ::end(data.bar));
What am I missing here?
Let's get a complete, reproducible example:
#include <iterator>
char* begin(char& c) { return &c; }
char* end(char& c) { return &c + 1; }
namespace ns {
void my_algorithm(char *first, char *last);
void my_function() {
using std::begin;
using std::end;
char c = '0';
my_algorithm(begin(c), end(c));
}
}
When you make the unqualified call to begin(c) and end(c), the compiler goes through the process of unqualified name lookup (described on the Argument-dependent lookup page of cppreference).
For regular unqualified name lookup, the process is roughly to start at the namespace you are currently in—::ns in this case—and only move out a namespace if you don't find the specific name.
If a function call is unqualified, as it is here with begin(c) and end(c), argument dependent lookup can occur, which finds free functions declared in the same namespace as the types of the functions' arguments, through the process of extending the overload set by finding "associated namespaces."
In this case, however, char is a fundamental type, so argument dependent lookup doesn't allow us to find the global ::begin and ::end functions.
For arguments of fundamental type, the associated set of namespaces and classes is empty
cppreference: argument dependent lookup
Instead, as we already have using std::begin; using std::end;, the compiler already sees possible functions for begin(...) and end(...)—namely those defined in namespace ::std—without having to move out a namespace from ::ns to ::. Thus, the compiler uses those functions, and compilation fails.
It's worth noting that the using std::begin; using std::end; also block the compiler from finding the custom ::begin and ::end even if you were to place them inside ::ns.
What you can do instead is write your own begin and end:
#include <iterator>
namespace ns {
char* begin(char& c) { return &c; }
char* end(char& c) { return &c + 1; }
template <typename T>
auto begin(T&& t) {
using std::begin;
// Not unbounded recursion if there's no `std::begin(t)`
// or ADL `begin(t)`, for the same reason that our
// char* begin(char& c); overload isn't found with
// using std::begin; begin(c);
return begin(t);
}
template <typename T>
auto end(T&& t) {
using std::end;
return end(t);
}
void my_algorithm(char *first, char *last);
void my_function() {
char c = '0';
my_algorithm(ns::begin(c), ns::end(c));
}
}
The title of question is "Overloading std::begin()". Overloading is possible only within the same scope. That is you can't overload names from different scopes. In another scope we can only make efforts to help lookup name. Essentially, here "using std::begin" declaration hides ::begin in question's code. See S.Lippman for reference:
functions that are members of two distinct namespaces do not overload one another.
Scope of a using Declaration. Names introduced in a using declaration obey normal scope rules.
Entities with the same name defined in an outer scope are hidden.
As soon as parameter is char and char is fundamental type - argument dependent lookup should not be taken into consideration - as mentioned in comments - there is no associated namespace with fundamental types.
Again, the question was: "What am I missing?" - therefore the answer is focused only on reasons - recommendations may be too broad.
struct S
{
vector<int> v;
void method()
{
begin(v);
}
};
The above code snippet compiles fine, because of ADL until I add
auto begin() { return begin(v); }
to the class declaration. At that point C++ forgets about ADL and instead prefers S::begin that doesn't even have a viable overload, producing the error
error: no matching function for call to ‘S::begin(std::vector<int>&)’ begin(v);
Is there any fix to this? I am asking, because after reading Why use non-member begin and end functions in C++11?, I started using begin() and end() free functions everywhere for consistency, but now I am getting conflicts after defining my own begin() and end() methods.
As mentioned in comments S::begin hides std::begin. You can bring std::begin into S's scope by typing using std::begin or explicitly call std::begin.
struct S
{
std::vector<int> v;
void method()
{
using std::begin;
begin(v);
}
auto begin() { using std::begin; return begin(v); }
};
You're using begin as an unqualified name, and unqualified name lookup
... examines the scopes as described below, until it finds at least one declaration of any kind, at which time the lookup stops and no further scopes are examined.
reference.
From your member function's perspective, the first scope providing a name begin is the class scope, so it populates the overload set from that scope, and stops looking.
Only after this name lookup stage, does it try to choose one of the overload set, and decide nothing matches. The compiler doesn't then go back and start searching from the next scope, it just gives up.
Your options are:
use the existing container member function instead of the free function (this is slightly less verbose than the explicitly-qualified version below)
auto begin() { return v.begin(); }
use qualified names instead
auto begin() { return ::std::begin(v); }
add the correct overload to class scope, with using std::begin;
Ignore that one, I forgot you can't introduce non-member names at class scope with using.
inject the correct overload into the narrower scope of the member function body itself, so the search stops there
auto begin() { using std::begin; return begin(v); }
stop providing a begin member function in the first place, and instead add a non-member begin overload to the enclosing namespace. This is more modern and avoids the lookup problem.
namespace N {
struct S { std::vector<int> v; };
std::vector<int>::iterator begin(S& s) { return s.v.begin(); }
// and end, cbegin, etc. etc.
}
I'm trying to specialize std::begin for a custom container. I'm doing this because I want to use range-based for with the container. This is what I have:
class stackiterator { … };
class stack { … };
#include <iterator>
template <> stackiterator std::begin(stack& S)
{
return S.GetBottom();
}
I get the following error at the definition of my begin specialization:
No function template matches function template specialization 'begin'
What am I doing wrong?
I'm trying to specialize std::begin for a custom container. I'm
doing this because I want to use range-based for with the container.
You are barking up the wrong tree. Range-based for does not use std::begin at all. For class types, the compiler looks up member begin and end directly, and if neither is found, does an ADL-only lookup for free begin and end in the associated namespaces. Ordinary unqualified lookup is not performed; there's no way for std::begin to be picked up if your class is not in the std namespace.
Even if the specialization you want to do is possible (it isn't unless you introduce a member begin() - an explicit specialization for a function template can't change the return type, and the overload at issue returns "whatever member begin() returns"; and if you do introduce a member begin(), why are you specializing std::begin to do what it would have done anyway?), you still won't be able to use it with a range-based for.
Leaving aside the policy and semantic issues of whether you should specialize a function template from the std namespace,
The following snippet does not work:
class stackiterator {};
struct stack { stackiterator Begin() { return stackiterator{};} };
#include <iterator>
namespace std
{
template <> stackiterator begin<stack>(stack& S)
{
return S.Begin();
}
}
However, the following snippet works just fine:
class stackiterator {};
struct stack { stackiterator begin() { return stackiterator{};} };
#include <iterator>
namespace std
{
template <> stackiterator begin<stack>(stack& S)
{
return S.begin();
}
}
The key difference is the presence of Begin() vs begin() as a member function of stack. std::begin() is defined as:
template <class C> auto begin(C& c) -> decltype(c.begin());
template <class C> auto begin(const C& c) -> decltype(c.begin());
When you specialize a function template, you must still keep the return type the same. When you don't have begin() as a member of Stack, the compiler does not know how to determine the return type.
That is the reason for error produced by the compiler.
BTW, there is another SO post that partially answers what can be specialized and what can't be specialized.
Looking at the part of the standard that deals with std::begin(), Section 24.3, I don't see anything about not being able to specialize std::begin().
The right way to add a free function begin that enables for(:) loops is to add, in the namespace of stack, a begin(stack&) and begin(stack const&) function that returns a iterator and const_iterator respectively (and ditto for end)
The other way is to add a member begin() and end() to stack.
Specializing std::begin is bad practice for a number of reasons, not the least of which is that not all for(:) loops will work with it (the lookup rules where changed in the resolution of this defect report). Overloading std::begin is undefined behavior (you may not overload functions in namespace std under the standard: doing so makes your program ill-formed).
This is how it has to be done, even if it violates the naming convention of your project.
The <algorithm> header provides std::equal_range(), as well as some containers having it as a member function. What bothers me with this function is that it returns a pair of iterators, making it tedious to iterate from the begin iterator to the end iterator. I'd like to be able to use std::begin() and std::end() so that I can use the C++11 range-based for-loop.
Now, I've heard contradictory information in regards to specializing std::begin() and std::end() - I've been told that adding anything to the std namespace results in undefined behavior, whereas I have also been told that you can provide your own specializations of std::begin() and std::end().
This is what I am doing right now:
namespace std
{
template<typename Iter, typename = typename iterator_traits<Iter>::iterator_category>
Iter begin(pair<Iter, Iter> const &p)
{
return p.first;
}
template<typename Iter, typename = typename iterator_traits<Iter>::iterator_category>
Iter end(pair<Iter, Iter> const &p)
{
return p.second;
}
}
And this does work: http://ideone.com/wHVfkh
But I am wondering, what are the downsides to doing this? Is there a better way to do this?
17.6.4.2.1/1 The behavior of a C++ program is undefined if it adds declarations or definitions to namespace std or to a namespace
within namespace std unless otherwise specified. A program may add a
template specialization for any standard library template to namespace
std only if the declaration depends on a user-defined type and the
specialization meets the standard library requirements for the
original template and is not explicitly prohibited.
So yes, I believe that, technically, your code exhibits undefined behavior. Perhaps you can write a simple class that takes a pair of iterators in its constructor and implements begin() and end() methods. Then you can write something like
for (const auto& elem: as_range(equal_range(...))) {}