I have seen multiple instances of code where function parameter pack is declared using the && notation, as shown below, but I cannot see any advantage to using this notation.
template<typename... Args>
void Function(Args... args)
{
}
template<typename... Args>
void Function(Args&&... args)
{
}
My first thought was that the && form will be used exclusively for r-value objects, but this test proved that wrong:
struct Object
{
// Added bodies so I see what is being called via a step-into
Object() {}
Object(const Object&) {}
Object(Object&&) noexcept {}
Object& operator=(const Object&) { return *this; }
Object& operator=(Object&&) noexcept { return *this; }
};
Object GetObject() { Object o; return o; }
Object obj;
Function(GetObject());
Function(GetObject());
Here, VS 2017 complains that both forms of the function are viable candidates for the call.
Can someone explain what the difference is between these two, and what advantages one may have over the other please?
They are forwarding references in the parameter pack form. As for template parameter deduction, they can match any arguments, but the template parameter will be deduced differently comparing to the ordinary template parameter.
The major advantage of forwarding reference is that the lvalue/rvalue information will be preserved if used with std::forward. Thus they are used to "forward" something.
For example,
void real_foo(A const &a);
void real_foo(A &&a);
template<class... Args>
void foo_proxy_ordinary(Args... args) { real_foo(args...); }
template<class... Args>
void foo_proxy_perfect(Args&&... args) { real_foo(std::forward<Args>(args)...); }
The ordinary version will always call real_foo(A const &) version, because inside foo_proxy, args are always lvalue.
However, the perfect version will select real_foo(A&&) if the arguments passed in are indeed rvalues.
Combining forwarding reference with parameter pack, one can write easily generic proxy functions without performance loss in terms of lvalue/rvalue.
T&& when used in the context of
template<typename T>
void f(T&& t);
is called a forwarding reference sometimes also called a universal reference.
Main advantage of a forwarding reference is that combined with std::forward it enables achieving a so-called perfect forwarding: function template passing its arguments to another function as they are (lvalue as lvalue, rvalue as rvalue).
Now it is possible to create higher-order functions that take other functions as arguments or return them, or superior function-wrappers (e.g., std::make_shared), and do other cool things.
Here is some material that explains it much better and in more detail than I possibly can:
Perfect forwarding and universal references in C++
Rvalue References and Perfect Forwarding in C++0x
Forwarding references proposal
SO: Advantages of using forward
SO: Perfect forwarding - what's it all about?
Can someone explain what the difference is between these two, and what advantages one may have over the other please?
The difference is same for parameter packs as it is for individual parameters. Args declares an "object parameter" (pass by value) and Args&& declares a reference parameter (pass by reference).
Passing by reference allows one to avoid copying the argument when that is unnecessary. It also allows modifying the referred argument if the reference is non-const, which includes the possibility of moving from that object.
Passing by value makes it clear to the caller that the passed object will neither be modified, nor be referred to as a result of calling the function.
My first thought was that the && form will be used exclusively for r-value objects
As your test demonstrates, that is indeed an incorrect assumption. When Args is a deduced type i.e. auto or a template argument, Args&& can indeed be either an l-value reference or an r-value reference. Which one it is depends on what Args is deduced to be. This demonstrates the reference collapsing rules concisely:
typedef int& lref;
typedef int&& rref;
int n;
lref& r1 = n; // type of r1 is int&
lref&& r2 = n; // type of r2 is int& note this case in particular
rref& r3 = n; // type of r3 is int&
rref&& r4 = 1; // type of r4 is int&&
Using such reference allows forwarding i.e. re-binding into a new lvalue reference (when possible) or moving from the object (when possible) or copying (when neither of the previous is possible).
Because of this, Args&& is called a forwarding reference (or a universal reference) when Args is a deduced type.
Related
The standard signature of std::forward is:
template<typename T>
constexpr T&& forward(std::remove_reference_t<T>&) noexcept;
template<typename T>
constexpr T&& forward(std::remove_reference_t<T>&&) noexcept;
Because the parameter type isn't T directly, we should specify the template argument when using std::forward:
template<typename... Args>
void foo(Args&&... args)
{
bar(std::forward<Args>(args)...);
}
However sometimes the template argument is not as simple as Args. auto&& is a case:
auto&& vec = foo();
bar(std::forward<decltype(vec)>(vec));
You can also imagine more complicated template argument types for std::forward. Anyway, intuitively std::forward should know what T is but it actually don't.
So my idea is to omit <Args> and <decltype(vec)> no matter how simple they are. Here is my implementation:
#include <type_traits>
template<typename T>
std::add_rvalue_reference_t<std::enable_if_t<!std::is_lvalue_reference<T>::value, T>>
my_forward(T&& obj)
{
return std::move(obj);
}
template<typename T>
T& my_forward(T& obj)
{
return obj;
}
int main()
{
my_forward(1); // my_forward(int&&)
int i = 2;
my_forward(i); // my_forward(int&)
const int j = 3;
my_forward(j); // my_forward(const int&)
}
When obj is rvalue reference, for example int&&, the first overload is selected because T is int, whose is_lvalue_reference is false;
When obj is lvalue reference, for example const int&, the second overload is selected because T is const int& and the first is SFINAE-ed out.
If my implementation is feasible, why is std::forward still requiring <T>? (So mine must be infeasible.)
If not, what's wrong? And still the question, is it possible to omit template parameter in std::forward?
The problematic case is when you pass something of rvalue reference type but which does not belong to an rvalue value category:
int && ir{::std::move(i)};
my_forward(ir); // my_forward(int&)
Passing type to std::forward will ensure that arguments of rvalue reference types will be moved further as rvalues.
The answer by user7860670 gives you an example for the case where this breaks down. Here is the reason why the explicit template parameter is always needed there.
By looking at the value of the forwarding reference you can no longer reliably determine through overload resolution whether it is safe to move from. When passing an lvalue reference parameter as an argument to a nested function call it will be treated as an lvalue. In particular, it will not bind as an rvalue argument, that would require an explicit std::move again. This curious asymmetry is what breaks implicit forwarding.
The only way to decide whether the argument should be moved onwards is by inspecting its original type. But the called function cannot do so implicitly, which is why we must pass the deduced type explicitly as a template parameter. Only by inspecting that type directly can we determine whether we do or do not want to move for that argument.
I was looking at the std::function implementation and its call operator()
template<typename Ret, typename... ArgTypes>
Ret function< Ret (ArgTypes...)>::operator()(ArgTypes...args) const
{
// some stuff
return invoker(functor, std::forward<ArgTypes>(args)...);
}
I was particularly wondering, why it uses std::forward here? Does this anything have to do with perfect forwarding?
Because perfect forwarding could only be done if operator() is a template with a variadic template declaration template<typename... Args> (which it is not, the declaration is a partial specialization of std::function).
What is the intention of using std::forward here? I am confused :-)?
You are correct that this is not your typical "perfect forwarding" scenario. A brief example can help illustrate the motivation. Assume a type A with instrumented constructors and destructor:
#include "A.h"
#include <functional>
#include <iostream>
int
main()
{
A a1{1};
A a2{2};
std::function<void(A, A&)> f{[](A x, A& y){}};
f(a1, a2);
}
This will output:
A(int state): 1
A(int state): 2
A(A const& a): 1
A(A&& a): 1
~A(1)
~A(-1)
~A(2)
~A(1)
Explanation:
a1 and a2 are constructed on the stack. Then when passed into the function invoker, a1 is first copied to bind to the first by-value parameter, and then std::forward<A> is called on a1 which moves it from the by-value parameter into the lambda.
In contrast, a2 need not be copied to bind to the function A& parameter, and then std::forward<A&>(a2) is called, which forwards a2 as an lvalue instead of rvalue, and this binds to the A& parameter of the lambda.
Then things get destructed. The ~A(-1) indicates the destruction of an A in a move-constructed-from state with this instrumented A.
In summary, even though ArgTypes isn't deduced as in the usual perfect forwarding idiom, we still want to forward by-value ArgTypes as rvalues, and by-reference ArgTypes as lvalues. So std::forward just happens to do exactly what we want here.
I think you are confused by many things here.
First, perfect forwarding has nothing to do with variadic templates. You could create a wrapper class that has a function that takes one argument and forward it to the wrapped object :
template<typename T>
struct Wrapper {
template<typename Arg>
decltype(auto) test(Arg&& arg) {
return t.test(std::forward<Arg>(arg));
}
T t;
};
Notice the use of perfect forwarding here without any variadic templates. If t.test would require a move only type as parameter, it would not be possible to call it without the forward<Arg>(arg).
The second thing happening here is the parameter not being followed by &&. Adding && to ArgTypes would be a mistake and would make some cases fail to compile. Consider this simple case :
std::function<void(int)> f;
int i = 0;
f(i);
That would fail to compile. If you add && to ArgTypes, every parameters that are not reference (eg. int) would become an rvalue reference on the call operator (in our case, int&&). Since all parameter types are already qualified correctly in the std::function argument list, what you want to recieve in the call operator is exactly those types, not transformed.
The why you need std::forward if you don't use &&? Because even though you don't need to infer value categories, you still need to not copy every arguments to the contained function. If one of the std::function's parameter is int&, you don't want to move it. But if one of the parameter is std::unique_ptr<int>, you must move it! And this is exactly what std::forward is for. Moving only what should be moved.
std::forward just appends rvalue reference to the type, so taking reference collapsing rules into account, it effectively passes reference arguments as-is and moves the object arguments.
I know that this can be used to perform perfect forwarding:
template <typename A>
void foo(A&&) { /* */ }
This can be used to perform perfect forwarding on a certain type:
template <typename A, std::enable_if_t<std::is_same<std::decay_t<A>, int>::value, int> = 0>
void foo(A&&) { /* */ }
But these are just templates for functions, which means, that these get expanded to some functions, which are then used for every special case in which it might be used. However do these get expanded to:
void foo(A&) and void foo(A&&)
OR
void foo(A&) and void foo(A)
I always thought, it would be the first one, but then I noticed, that in that case, you wouldn't be able to use A const as an argument to the function, which certainly works.
However the second would be ambiguous, if you used a normal non-const lvalue. Does it call foo(A&) or foo(A)?
It's the first one. The second wouldn't make very much sense: there is no A such that A&& is a non-reference type.
If the argument is an lvalue of type cv T, then A is deduced as cv T&. If the argument is an rvalue of type cv T, then A is deduced as cv T and A&& is cv T&&. So when you pass in a const lvalue, the specialization generated is one that can accept a const argument.
They were called originally "Univeral References" by Scott Meyers, and now "Forwarding References".
As you can see, the references part has not changed. You pass in any kind of rvalue, you get a rvalue reference. You pass in any kind of lvalue, and you get a lvalue reference. Life is that simple.
An example here for std::forward,
// forward example
#include <utility> // std::forward
#include <iostream> // std::cout
// function with lvalue and rvalue reference overloads:
void overloaded (const int& x) {std::cout << "[lvalue]";}
void overloaded (int&& x) {std::cout << "[rvalue]";}
// function template taking rvalue reference to deduced type:
template <class T> void fn (T&& x) {
overloaded (x); // always an lvalue
overloaded (std::forward<T>(x)); // rvalue if argument is rvalue
}
int main () {
int a;
std::cout << "calling fn with lvalue: ";
fn (a);
std::cout << '\n';
std::cout << "calling fn with rvalue: ";
fn (0);
std::cout << '\n';
return 0;
}
Output:
calling fn with lvalue: [lvalue][lvalue]
calling fn with rvalue: [lvalue][rvalue]
mentions that
the fact that all named values (such as function parameters) always
evaluate as lvalues (even those declared as rvalue references)
Whereas, the typical move constructor looks like
ClassName(ClassName&& other)
: _data(other._data)
{
}
which looks like _data(other._data) should invoke the move constructor of _data's class. But, how is it possible without using std::forward? In other words, shouldn't it be
ClassName(ClassName&& other)
: _data(std::forward(other._data))
{
}
?
Because, as pointed out in std:forward case,
all then named values should evaluate as lvalue
I more and more like C++ because of the depth of issue like this and the fact that the language is bold enough to provide such features :) Thank you!
A typical move constructor looks like this (assuming it is explicitly implemented: you might want to prefer = default):
ClassName::ClassName(ClassName&& other)
: _data(std::move(other._data)) {
}
Without the std::move() the member is copied: since it has a name other is an lvalue. The object the reference is bound to is an rvalue or an object considered as such, however.
std::forward<T>(obj) is always used with an explicit template argument. In practice the type is that deduced for a forwarding reference. These look remarkably like rvalue references but are something entirely different! In particular, a forwarding reference may refer to an lvalue.
You may be interested in my Two Daemons article which describes the difference in detail.
std::forward should be used with a forwarding reference.
std::move should be used with an rvalue reference.
There is nothing particular about constructors. The rules apply the same to any function, member function or constructor.
The most important thing is to realize when you have a forwarding reference and when you have an rvalue reference. They look similar but are not.
A forwarding reference is always in the form:
T&& ref
for T some deduced type.
For instance, this is a forwarding reference:
template <class T>
auto foo(T&& ref) -> void;
All these are rvalue references:
auto foo(int&& ref) -> void; // int not deduced
template <class T>
auto foo(const T&& ref); // not in form `T&&` (note the const)
template <class T>
auto foo(std::vector<T>&& ref) -> void; // not in form `T&&`
template <class T>
struct X {
auto foo(T&& ref) -> T; // T not deduced. (It was deduced at class level)
};
For more please check this excellent in-depth article by Scott Meyers with the note that when the article was written the term "universal reference" was used (actually introduced by Scott himself). Now it is agreed that "forwarding reference" better describes it's purpose and usage.
So your example should be:
ClassName(ClassName&& other)
: _data(std::move(other._data))
{
}
as other is an rvalue reference because ClassName is not a deduced type.
This Ideone example should make things pretty clear for you. If not, keep reading.
The following constructor accepts Rvalues only. However, since the argument "other" got a name it lost its "rvalueness" and now is a Lvalue. To cast it back to Rvalue, you have to use std::move. There's no reason to use std::forward here because this constructor does not accept Lvalues. If you try to call it with a Lvalue, you will get compile error.
ClassName(ClassName&& other)
: _data(std::move(other._data))
{
// If you don't use move, you could have:
// cout << other._data;
// And you will notice "other" has not been moved.
}
The following constructor accepts both Lvalues and Rvalues. Scott Meyers called it "Universal Rerefences", but now it's called "Forwarding References". That's why, here, it's a must to use std::forward so that if other was an Rvalue, _data constructor will get called with an Rvalue. If other was an Lvalue, _data will be constructed with an Lvalue. That's why it's called perfect-forwarding.
template<typename T>
ClassName(T&& other)
: _data(std::forward<decltype(_data)>(other._data))
{
}
I've tried to use your constructors as an example so you could understand, but this is not specific to constructors. This applies to functions as well.
With the first example tho, since your first constructor only accepts Rvalues, you could perfectly use std::forward instead, and both would do the same thing. But it's best not to do it, because people may think that your constructor accepts a forwarding reference, when it actually doesn't.
C++11 (and C++14) introduces additional language constructs and improvements that target generic programming. These include features such as;
R-value references
Reference collapsing
Perfect forwarding
Move semantics, variadic templates and more
I was browsing an earlier draft of the C++14 specification (now with updated text) and the code in an example in ยง20.5.1, Compile-time integer sequences, that I found interesting and peculiar.
template<class F, class Tuple, std::size_t... I>
decltype(auto) apply_impl(F&& f, Tuple&& t, index_sequence<I...>) {
return std::forward<F>(f)(std::get<I>(std::forward<Tuple>(t))...);
}
template<class F, class Tuple>
decltype(auto) apply(F&& f, Tuple&& t) {
using Indices = make_index_sequence<std::tuple_size<Tuple>::value>;
return apply_impl(std::forward<F>(f), std::forward<Tuple>(t), Indices());
}
Online here [intseq.general]/2.
Question
Why was the function f in apply_impl being forwarded, i.e. why std::forward<F>(f)(std::get...?
Why not just apply the function as f(std::get...?
In Brief...
The TL;DR, you want to preserve the value category (r-value/l-value nature) of the functor because this can affect the overload resolution, in particular the ref-qualified members.
Function definition reduction
To focus on the issue of the function being forwarded, I've reduced the sample (and made it compile with a C++11 compiler) to;
template<class F, class... Args>
auto apply_impl(F&& func, Args&&... args) -> decltype(std::forward<F>(func)(std::forward<Args>(args)...)) {
return std::forward<F>(func)(std::forward<Args>(args)...);
}
And we create a second form, where we replace the std::forward(func) with just func;
template<class F, class... Args>
auto apply_impl_2(F&& func, Args&&... args) -> decltype(func(std::forward<Args>(args)...)) {
return func(std::forward<Args>(args)...);
}
Sample evaluation
Evaluating some empirical evidence of how this behaves (with conforming compilers) is a neat starting point for evaluating why the code example was written as such. Hence, in addition we will define a general functor;
struct Functor1 {
int operator()(int id) const
{
std::cout << "Functor1 ... " << id << std::endl;
return id;
}
};
Initial sample
Run some sample code;
int main()
{
Functor1 func1;
apply_impl_2(func1, 1);
apply_impl_2(Functor1(), 2);
apply_impl(func1, 3);
apply_impl(Functor1(), 4);
}
And the output is as expected, independent of whether an r-value is used Functor1() or an l-value func when making the call to apply_impl and apply_impl_2 the overloaded call operator is called. It is called for both r-values and l-values. Under C++03, this was all you got, you could not overload member methods based on the "r-value-ness" or "l-value-ness" of the object.
Functor1 ... 1
Functor1 ... 2
Functor1 ... 3
Functor1 ... 4
Ref-qualified samples
We now need to overload that call operator to stretch this a little further...
struct Functor2 {
int operator()(int id) const &
{
std::cout << "Functor2 &... " << id << std::endl;
return id;
}
int operator()(int id) &&
{
std::cout << "Functor2 &&... " << id << std::endl;
return id;
}
};
We run another sample set;
int main()
{
Functor2 func2;
apply_impl_2(func2, 5);
apply_impl_2(Functor2(), 6);
apply_impl(func2, 7);
apply_impl(Functor2(), 8);
}
And the output is;
Functor2 &... 5
Functor2 &... 6
Functor2 &... 7
Functor2 &&... 8
Discussion
In the case of apply_impl_2 (id 5 and 6), the output is not as may have been initially been expected. In both cases, the l-value qualified operator() is called (the r-value is not called at all). It may have been expected that since Functor2(), an r-value, is used to call apply_impl_2 the r-value qualified operator() would have been called. The func, as a named parameter to apply_impl_2, is an r-value reference, but since it is named, it is itself an l-value. Hence the l-value qualified operator()(int) const& is called in both the case of the l-value func2 being the argument and the r-value Functor2() being used as the argument.
In the case of apply_impl (id 7 and 8) the std::forward<F>(func) maintains or preserves the r-value/l-value nature of the argument provided for func. Hence the l-value qualified operator()(int) const& is called with the l-value func2 used as the argument and the r-value qualified operator()(int)&& when the r-value Functor2() is used as the argument. This behaviour is what would have been expected.
Conclusions
The use of std::forward, via perfect forwarding, ensures that we preserve the r-value/l-value nature of the original argument for func. It preserves their value category.
It is required, std::forward can and should be used for more than just forwarding arguments to functions, but also when the use of an argument is required where the r-value/l-value nature must be preserved. Note; there are situations where the r-value/l-value cannot or should not be preserved, in these situations std::forward should not be used (see the converse below).
There are many examples popping up that inadvertently lose the r-value/l-value nature of the arguments via a seemingly innocent use of an r-value reference.
It has always been hard to write well defined and sound generic code. With the introduction of r-value references, and reference collapsing in particular, it has become possible to write better generic code, more concisely, but we need to be ever more aware of what the original nature of the arguments provided are and make sure that they are maintained when we use them in the generic code we write.
Full sample code can be found here
Corollary and converse
A corollary of the question would be; given reference collapsing in a templated function, how is the r-value/l-value nature of the argument maintained? The answer - use std::forward<T>(t).
Converse; does std::forward solve all your "universal reference" problems? No it doesn't, there are cases where it should not be used, such as forwarding the value more than once.
Brief background to perfect forwarding
Perfect forwarding may be unfamiliar to some, so what is perfect forwarding?
In brief, perfect forwarding is there to ensure that the argument provided to a function is forwarded (passed) to another function with the same value category (basically r-value vs. l-value) as originally provided. It is typically used with template functions where reference collapsing may have taken place.
Scott Meyers gives the following pseudo code in his Going Native 2013 presentation to explain the workings of std::forward (at approximately the 20 minute mark);
template <typename T>
T&& forward(T&& param) { // T&& here is formulated to disallow type deduction
if (is_lvalue_reference<T>::value) {
return param; // return type T&& collapses to T& in this case
}
else {
return move(param);
}
}
Perfect forwarding depends on a handful of fundamental language constructs new to C++11 that form the bases for much of what we now see in generic programming:
Reference collapsing
Rvalue references
Move semantics
The use of std::forward is currently intended in the formulaic std::forward<T>, understanding how std::forward works helps understand why this is such, and also aids in identifying non-idiomatic or incorrect use of rvalues, reference collapsing and ilk.
Thomas Becker provides a nice, but dense write up on the perfect forwarding problem and solution.
What are ref-qualifiers?
The ref-qualifiers (lvalue ref-qualifier & and rvalue ref-qualifier &&) are similar to the cv-qualifiers in that they (the ref-qualified members) are used during overload resolution to determine which method to call. They behave as you would expect them to; the & applies to lvalues and && to rvalues. Note: Unlike cv-qualification, *this remains an l-value expression.
Here is a practical example.
struct concat {
std::vector<int> state;
std::vector<int> const& operator()(int x)&{
state.push_back(x);
return state;
}
std::vector<int> operator()(int x)&&{
state.push_back(x);
return std::move(state);
}
std::vector<int> const& operator()()&{ return state; }
std::vector<int> operator()()&&{ return std::move(state); }
};
This function object takes an x, and concatenates it to an internal std::vector. It then returns that std::vector.
If evaluated in an rvalue context it moves to a temporary, otherwise it returns a const& to the internal vector.
Now we call apply:
auto result = apply( concat{}, std::make_tuple(2) );
because we carefully forwarded our function object, only 1 std::vector buffer is allocated. It is simply moved out to result.
Without the careful forwarding, we end up creating an internal std::vector, and we copy it to result, then discard the internal std::vector.
Because the operator()&& knows that the function object should be treated as a rvalue about to be destroyed, it can rip the guts out of the function object while doing its operation. The operator()& cannot do this.
Careful use of perfect forwarding of function objects enables this optimization.
Note, however, that there is very little use of this technique "in the wild" at this point. Rvalue qualified overloading is obscure, and doing so to operator() moreso.
I could easily see future versions of C++ automatically using the rvalue state of a lambda to implicitly move its captured-by-value data in certain contexts, however.