This is a question on lambda overload sets and perfect forwarding and somewhat of a followup to a comment. For more context of how this is used see another related question.
I have some questions on the below code snippet.
Q1: For lambda overloads, I was using overload(Fs...) -> overload<Fs...> from this post, but then in this answer I saw overload(Fs&&...) -> overload<std::decay_t<Fs>...>. In what situations is this difference relevant?
Q2: Why would you want to define the identity function below with return decltype(x)(x) and not just return x?
Q3: Can we consider foo(convert(std::forward<Args>(args))...) as perfect forwarding (for all not-converted arguments) just like foo(std::forward<Args>(args)...)?
#include <utility>
#include <iostream>
/////////////////////////////////////////////////////////////////////////////////
struct Foo {
virtual ~Foo() = default;
};
struct FooA: public Foo {
static void foo(const FooA&, int) { std::cout << __PRETTY_FUNCTION__ << std::endl; }
};
struct FooB: public Foo {
static void foo(int, const FooB&) { std::cout << __PRETTY_FUNCTION__ << std::endl; }
};
/////////////////////////////////////////////////////////////////////////////////
template<class...Fs>
struct overload:Fs... {
using Fs::operator()...;
};
// Q1: In what situations is needed over `overload(Fs...) -> overload<Fs...>`?
template<class...Fs>
overload(Fs&&...) -> overload<std::decay_t<Fs>...>;
/////////////////////////////////////////////////////////////////////////////////
// Q2: What is the purpose of `return decltype(x)(x)` over `return x`?
auto identity=[](auto&&x)->decltype(x){return decltype(x)(x);};
template<typename SpecificFoo, typename... Args>
void bar(Args&&... args) {
auto convert = overload{
[](const Foo& f){return dynamic_cast<const SpecificFoo&>(f);},
identity
};
// Q3: Is our definition of `convert` "perfectly forwarding", like if we just called
// `foo(std::forward<Args>(args)...)`, or in what situations might this not do the
// same thing (for not-converted types)?
SpecificFoo::foo(convert(std::forward<Args>(args))...);
}
/////////////////////////////////////////////////////////////////////////////////
int main() {
{
FooA specific_foo;
const Foo& foo {specific_foo};
// assume we only have access to foo when calling bar
bar<FooA>(foo, 23);
}
{
FooB specific_foo;
const Foo& foo {specific_foo};
// assume we only have access to foo when calling bar
bar<FooB>(42, foo);
}
}
run it
In what situations is this difference relevant?
When at least one argument is in fact an lvalue (like identity, in fact). In which case the corresponding Fi is a T&, i.e. an lvalue reference. And one can't list an lvalue reference as a base of any class, so std::decay is required to remove all reference and cv-qualifiers. When the deduction guide takes arguments by value, it's automatically a non-issue. This is because template argument deduction for value types already "decays" the types.
If you wonder which one to use, then I'd say the one with less clutter is objectively better. The use of std::decay_t is for getting the same behavior one would get with the by-value version, so one may as well use that.
Why would you want to define the identity function below with return decltype(x)(x) and not just return x
This is a form of forwarding. Since the return type of the lambda is declared to be decltype(x), we need the cast to make sure it binds correctly to an rvalue reference. Because in the case decltype(x) = T&&, it will not bind to x alone, which is an lvalue.
Can we consider foo(convert(std::forward<Args>(args))...) as perfect forwarding (for all not-converted arguments) just like foo(std::forward<Args>(args)...)
Yes. The arguments to bar already bound to references. convert lets those references pass through with value category preserved, so it's forwarding indeed.
I have to store arguments (parameter pack), and pass the arguments to another function.
As a result, I cannot use lambda. And a good choice is std::bind.
But for this code
struct A{};
void test(A &&){}
int main()
{
A a;
test(move(a)); //work
bind(test,a)(); //compile fail; copy a to std::bind, pass a to test
}
According to standard, all variables stored in std::bind will be pass as lvalue to function. (The C++ standard doesn't say that, by I think that is what it means.)
And that means I cannot use a function (has rvalue reference in parameter) with std::bind.
One solution is to change test(A &&) to test(A &), but this only works for your project (and make it strange while you not only need to call test by std::thread but also need to call test by plain sequential call).
So, is there any ways to solve this problem?
You can create wrapper which will be convertible to the rvalue reference (like reference_wrapper/l-value references) and use it with bind:
It cal look like that:
#include <iostream>
#include <functional>
struct A{};
void test(A &&){ std::cout << "Works!\n"; }
template <typename T>
struct rvalue_holder
{
T value;
explicit rvalue_holder(T&& arg): value(arg) {}
operator T&&()
{
return std::move(value);
}
};
template <typename T>
rvalue_holder<T> rval(T && val)
{
return rvalue_holder<T>(std::move(val));
}
int main()
{
A a;
test(std::move(a)); //work
auto foo = std::bind(test, rval(std::move(a))); //works
foo();
}
http://coliru.stacked-crooked.com/a/56220bc89a32c860
Note: both rvalue_holder and especially rval need further work to ensure efficiency, robustness and desired behavior in all cases.
Imagine you have a number of overloaded methods that (before C++11) looked like this:
class MyClass {
public:
void f(const MyBigType& a, int id);
void f(const MyBigType& a, string name);
void f(const MyBigType& a, int b, int c, int d);
// ...
};
This function makes a copy of a (MyBigType), so I want to add an optimization by providing a version of f that moves a instead of copying it.
My problem is that now the number of f overloads will duplicate:
class MyClass {
public:
void f(const MyBigType& a, int id);
void f(const MyBigType& a, string name);
void f(const MyBigType& a, int b, int c, int d);
// ...
void f(MyBigType&& a, int id);
void f(MyBigType&& a, string name);
void f(MyBigType&& a, int b, int c, int d);
// ...
};
If I had more parameters that could be moved, it would be unpractical to provide all the overloads.
Has anyone dealt with this issue? Is there a good solution/pattern to solve this problem?
Thanks!
Herb Sutter talks about something similar in a cppcon talk
This can be done but probably shouldn't. You can get the effect out using universal references and templates, but you want to constrain the type to MyBigType and things that are implicitly convertible to MyBigType. With some tmp tricks, you can do this:
class MyClass {
public:
template <typename T>
typename std::enable_if<std::is_convertible<T, MyBigType>::value, void>::type
f(T&& a, int id);
};
The only template parameter will match against the actual type of the parameter, the enable_if return type disallows incompatible types. I'll take it apart piece by piece
std::is_convertible<T, MyBigType>::value
This compile time expression will evaluate to true if T can be converted implicitly to a MyBigType. For example, if MyBigType were a std::string and T were a char* the expression would be true, but if T were an int it would be false.
typename std::enable_if<..., void>::type // where the ... is the above
this expression will result in void in the case that the is_convertible expression is true. When it's false, the expression will be malformed, so the template will be thrown out.
Inside the body of the function you'll need to use perfect forwarding, if you are planning on copy assigning or move assigning, the body would be something like
{
this->a_ = std::forward<T>(a);
}
Here's a coliru live example with a using MyBigType = std::string. As Herb says, this function can't be virtual and must be implemented in the header. The error messages you get from calling with a wrong type will be pretty rough compared to the non-templated overloads.
Thanks to Barry's comment for this suggestion, to reduce repetition, it's probably a good idea to create a template alias for the SFINAE mechanism. If you declare in your class
template <typename T>
using EnableIfIsMyBigType = typename std::enable_if<std::is_convertible<T, MyBigType>::value, void>::type;
then you could reduce the declarations to
template <typename T>
EnableIfIsMyBigType<T>
f(T&& a, int id);
However, this assumes all of your overloads have a void return type. If the return type differs you could use a two-argument alias instead
template <typename T, typename R>
using EnableIfIsMyBigType = typename std::enable_if<std::is_convertible<T, MyBigType>::value,R>::type;
Then declare with the return type specified
template <typename T>
EnableIfIsMyBigType<T, void> // void is the return type
f(T&& a, int id);
The slightly slower option is to take the argument by value. If you do
class MyClass {
public:
void f(MyBigType a, int id) {
this->a_ = std::move(a); // move assignment
}
};
In the case where f is passed an lvalue, it will copy construct a from its argument, then move assign it into this->a_. In the case that f is passed an rvalue, it will move construct a from the argument and then move assign. A live example of this behavior is here. Note that I use -fno-elide-constructors, without that flag, the rvalue cases elides the move construction and only the move assignment takes place.
If the object is expensive to move (std::array for example) this approach will be noticeably slower than the super-optimized first version. Also, consider watching this part of Herb's talk that Chris Drew links to in the comments to understand when it could be slower than using references. If you have a copy of Effective Modern C++ by Scott Meyers, he discusses the ups and downs in item 41.
You may do something like the following.
class MyClass {
public:
void f(MyBigType a, int id) { this->a = std::move(a); /*...*/ }
void f(MyBigType a, string name);
void f(MyBigType a, int b, int c, int d);
// ...
};
You just have an extra move (which may be optimized).
My first thought is that you should change the parameters to pass by value. This covers the existing need to copy, except the copy happens at the call point rather than explicitly in the function. It also allows the parameters to be created by move construction in a move-able context (either unnamed temporaries or by using std::move).
Why you would do that
These extra overloads only make sense, if modifying the function paramers in the implementation of the function really gives you a signigicant performance gain (or some kind of guarantee). This is hardly ever the case except for the case of constructors or assignment operators. Therefore, I would advise you to rethink, whether putting these overloads there is really necessary.
If the implementations are almost identical...
From my experience this modification is simply passing the parameter to another function wrapped in std::move() and the rest of the function is identical to the const & version. In that case you might turn your function into a template of this kind:
template <typename T> void f(T && a, int id);
Then in the function implementation you just replace the std::move(a) operation with std::forward<T>(a) and it should work. You can constrain the parameter type T with std::enable_if, if you like.
In the const ref case: Don't create a temporary, just to to modify it
If in the case of constant references you create a copy of your parameter and then continue the same way the move version works, then you may as well just pass the parameter by value and use the same implementation you used for the move version.
void f( MyBigData a, int id );
This will usually give you the same performance in both cases and you only need one overload and implementation. Lots of plusses!
Significantly different implementations
In case the two implementations differ significantly, there is no generic solution as far as I know. And I believe there can be none. This is also the only case, where doing this really makes sense, if profiling the performance shows you adequate improvements.
You might introduce a mutable object:
#include <memory>
#include <type_traits>
// Mutable
// =======
template <typename T>
class Mutable
{
public:
Mutable(const T& value) : m_ptr(new(m_storage) T(value)) {}
Mutable(T& value) : m_ptr(&value) {}
Mutable(T&& value) : m_ptr(new(m_storage) T(std::move(value))) {}
~Mutable() {
auto storage = reinterpret_cast<T*>(m_storage);
if(m_ptr == storage)
m_ptr->~T();
}
Mutable(const Mutable&) = delete;
Mutable& operator = (const Mutable&) = delete;
const T* operator -> () const { return m_ptr; }
T* operator -> () { return m_ptr; }
const T& operator * () const { return *m_ptr; }
T& operator * () { return *m_ptr; }
private:
T* m_ptr;
char m_storage[sizeof(T)];
};
// Usage
// =====
#include <iostream>
struct X
{
int value = 0;
X() { std::cout << "default\n"; }
X(const X&) { std::cout << "copy\n"; }
X(X&&) { std::cout << "move\n"; }
X& operator = (const X&) { std::cout << "assign copy\n"; return *this; }
X& operator = (X&&) { std::cout << "assign move\n"; return *this; }
~X() { std::cout << "destruct " << value << "\n"; }
};
X make_x() { return X(); }
void fn(Mutable<X>&& x) {
x->value = 1;
}
int main()
{
const X x0;
std::cout << "0:\n";
fn(x0);
std::cout << "1:\n";
X x1;
fn(x1);
std::cout << "2:\n";
fn(make_x());
std::cout << "End\n";
}
This is the critical part of the question:
This function makes a copy of a (MyBigType),
Unfortunately, it is a little ambiguous. We would like to know what is the ultimate target of the data in the parameter. Is it:
1) to be assigned to an object that existing before f was called?
2) or instead, stored in a local variable:
i.e:
void f(??? a, int id) {
this->x = ??? a ???;
...
}
or
void f(??? a, int id) {
MyBigType a_copy = ??? a ???;
...
}
Sometimes, the first version (the assignment) can be done without any copies or moves. If this->x is already long string, and if a is short, then it can efficiently reuse the existing capacity. No copy-construction, and no moves. In short, sometimes assignment can be faster because we can skip the copy contruction.
Anyway, here goes:
template<typename T>
void f(T&& a, int id) {
this->x = std::forward<T>(a); // is assigning
MyBigType local = std::forward<T>(a); // if move/copy constructing
}
If the move version will provide any optimization then the implementation of the move overloaded function and the copy one must be really different. I don't see a way to get around this without providing implementations for both.
I'm new to rvalue references in C++ and want to learn how to use them in everyday life.
I've got 2 connected questions about popular usecases: using rvalue references with boost::in_place and boost::bind.
Using rvalue ref in boost::in_place
Consider a class, with constructor taking rvalue reference as a parameter:
struct A
: boost::noncopyable
{
A(int&&){}
};
Let's now try to create boost optional variable for this class:
void foo(int&& value)
{
boost::optional<A> opt;
// some code here
opt = boost::in_place(std::forward<int>(value)); //Error!
}
What is the right way to pass rvalue ref in such an example. Is there any solution like boost::reference_wrapper for rvalue refs?
passing bind functor to function object with rvalue reference
Another common usecase is assigning boost::bind functor object to boost::function object.
void foo(int&&)
{
}
void bar()
{
boost::function<void(int&&)> func;
int x = 0;
func = boost::bind(foo, std::move(x)); // Compilation error (a)
func = boost::bind(foo, _1); // Compilation error too (b)
}
I understand that instruction (a) could lead to undefined variable value after first invocation, but instruction (b) even has no such a problem. But how to write this code correctly?
boost in_places does not natively support move or r-value references. However, you can make a new class that inherits from boost::in_place_factory_base, and make your own. Or you could use an optional that supports emplace functions (like folly::Optional).
I'm not sure I recommend doing it, but you can make a generic overload in boost to handle it, something like this:
#include <boost/optional.hpp>
#include <boost/utility/in_place_factory.hpp>
#include <folly/ApplyTuple.h>
namespace boost
{
template<class ... As>
struct rvalue_in_place : public in_place_factory_base
{
mutable std::tuple<As...> tup_;
rvalue_in_place(As ... as)
: tup_(std::forward<As>(as)...)
{
}
template< class T >
void apply ( void* address ) const
{
auto make = [address](As ... as) { new (address) T(std::forward<As>(as)...); };
folly::applyTuple(make, tup_);
}
};
template <class ... As>
rvalue_in_place<As&&...> in_place(As && ... a)
{
return rvalue_in_place<As&&...>(std::forward<As>(a)...);
}
}
I made a finally simulator using lambda in C++11 as below:
#include <cstdio>
template<typename Functor>
struct Finalizer
{
Finalizer(Functor& func) : func_(func) {} // (1)
~Finalizer() { func_(); }
private:
Functor func_; // (2)
};
template<typename functor>
Finalizer<functor> finally(functor& func)
{
return Finalizer<functor>(func); (3)
}
int main()
{
int a = 20;
// print the value of a at the escape of the scope
auto finalizer = finally([&]{ printf("%d\n", a); }); // (4)
}
The code works as intended, but there is undesired copy ctor call (of lambda functor) at the ctor of Finalizer struct (1). (Thankfully, copy construction at the return statement in the finally function (3 -> 4) is avoided by RVO.)
Compiler does not eliminate the copy ctor call (at least in vc10 - gcc may optimize it), and if the type of the functor in Finalizer struct (2) is changed to reference it'll crash since the lambda argument at the finally call (4) is r-value.
Of course the code can be "optimized" like below
template<typename Functor>
struct Finalizer
{
Finalizer(Functor& func) : func_(func) {}
~Finalizer() { func_(); }
private:
Functor& func_;
};
int main()
{
int a = 20;
auto finalizer = [&]{ printf("%d\n", a); };
Finalizer<decltype(finalizer)> fin(finalizer);
}
No overhead, only a printf call is placed at the end of scope. But... I don't like it. :( I tried to wrap it with macro, but it needs to declare two "name" - one for lambda object, the other for finalizer object.
My objective is simple -
Every unnecessary performance overhead which can be avoided should be eliminated. Ideally, there should be no function call, every procedure should be inlined.
Keep the concise expression as its purpose of utility function. Use of macro is allowed, but discouraged.
Is there any solution to avoid it for this situation?
I presume lambdas have move constructors? If so, and if you will only ever use rvalues inside finally, then && and forward will move rather than copy.
#include <cstdio>
template<typename Functor>
struct Finalizer
{
Finalizer(Functor&& func) : func_(std::move(func)) {}
Finalizer(Functor const& func) : func_(func) {} // (1)
~Finalizer() { func_(); }
private:
Functor func_; // (2)
};
template<typename functor>
Finalizer<std::remove_reference<functor>::type> finally(functor&& func)
{
return Finalizer<std::remove_reference<functor>::type>(std::forward<functor>(func)); // (3)
}
int main()
{
int a = 20;
// print the value of a at the escape of the scope
auto finalizer = finally([&]{ printf("%d\n", a); }); // (4)
}
It should be possible to right something more intelligent that will work correctly with lvalues too, so that you're 'optimized' version will compile and will copy when it cannot move. In that case, I suggest you use something like Functor<std::remove_reference<functor>::type> to be sure that the Functor is of the right type, regardless of whether the parameters were passed around by & or && or whatever.
Perhaps accept the functor argument to the constructor as an rvalue reference?
I would simply allow the functions to be passed by value.
This is done all through out the STL algorithms.
Then, both ways of calling would be fine:
this
auto finalizer = finally([&]{ printf("%d\n", a); }); // this can be dangerous, if passed by reference to finally.
and this
auto finalizer = [&]{ printf("%d\n", a); };
Finalizer<decltype(finalizer)> fin(finalizer);
The reason for doing so, would be that functions should not be large.
I think STL algorithms follow the same reasoning when passing functions around.
Here is the full code
#include <cstdio>
template<typename Functor>
struct Finalizer
{
Finalizer(Functor func) : func_(func) {} /// pass by value
~Finalizer() { func_(); }
private:
Functor func_; //
};
template<typename functor>
Finalizer<functor> finally(functor func) /// pass by value
{
return Finalizer<functor>(func);
}
int main()
{
int a = 20;
// print the value of a at the escape of the scope
auto finalizer = finally([&]{ printf("%d\n", a); });
}