I have a generic function that invokes a callable it's handed; the moral equivalent of a call to std::invoke, except it's part of a coroutine:
// Wait for some condition to be true, then invoke f with the supplied
// arguments. The caller must ensure that all references remain valid
// until the returned task is done.
template <typename F, typename Args...>
Task<void> WaitForConditionAndInvoke(F&& f, Args&&... args) {
co_await SomeCondition();
std::invoke(f, std::forward<Args>(args)...);
}
Because of the requirement that the input arguments remain valid, it's not always legal to pass WaitForConditionAndInvoke a temporary like a lambda (unless e.g. the returned task is directly co_awaited as part of a single expression). This is true even if using unary + to convert a lambda with no bound state to a function pointer: the function pointer is itself a temporary, and asan seems to correctly complain about using it after it's destroyed.
My question is whether using a member function pointer is legal:
struct Object {
void SomeMethod();
};
// Some global object; we're not worried about the object's lifetime.
Object object;
Task<void> WaitForConditionAndInvokeOnGlobalObject() {
return WaitForConditionAndInvoke(&Object::SomeMethod, &object);
}
This seems to work fine, but it's unclear to me what the lifetime of the pointer that &Object::SomeMethod evaluates to is. Is this guaranteed to be a constant expression, i.e. not a temporary? What part of the standard covers this?
That WaitForConditionAndInvoke coroutine will be dangerous unless every argument including the functor f refers to an object with lifetime long enough. For example, WaitForConditionAndInvoke(std::abs, 1) has undefined behavior because of the object materialized to initialize a reference with the int prvalue expression 1. There is no difference per the Standard for constant expression arguments here, although a constant expression value could help compilers implement it in a way which "works" using a dead object's known value.
To fix this, you could have your function move every rvalue argument into a local object:
// All rvalue arguments are moved from.
// The caller must make sure all lvalue arguments remain valid until
// the returned task is done.
template <typename F, typename Args...>
Task<void> WaitForConditionAndInvoke(F&& f, Args&&... args) {
// local_f could be declared an lvalue reference or object,
// but not an rvalue reference:
F local_f(std::forward<F>(f));
// Similarly, the template arguments to tuple are lvalue references
// or object types.
std::tuple<Args...> local_args(std::forward<Args>(args)...);
co_await SomeCondition();
std::apply(std::forward<F>(local_f), std::move(local_args));
}
Or to be even safer, do as std::bind does, and move/copy everything. The calling code can specify that the functor or functor argument(s) should be treated as a reference with no move or copy using std::ref or std::cref. In fact, that implementation is just:
// All arguments are moved or copied from.
// A std::reference_wrapper can be used to skip the move/copy of
// the referenced object.
template <typename F, typename Args...>
Task<void> WaitForConditionAndInvoke(F&& f, Args&&... args) {
auto bound_f = std::bind(std::forward<F>(f), std::forward<Args>(args)...);
co_await SomeCondition();
bound_f();
}
As the commenters say, there's no lifetime issue here. &Object::SomeMethod is a constant expression, just like 42 or &WaitForConditionAndInvokeOnGlobalObject.
You might have confused yourself by naming your type struct Object, so the expression &Object::SomeMethod kind of looks like it involves an object... but it doesn't; Object is the name of a type. There is no object involved in that expression; it's a simple compile-time constant, just like offsetof(Object, SomeDataMember) or sizeof(Object) would be.
No object involved = no lifetime issues involved.
EDIT (thus completely changing my answer): Ah, aschepler is right, you're concerned with the lifetime of the thing-referred-to-by-F&& f in
template <typename F, typename Args...>
Task<void> WaitForConditionAndInvoke(F&& f, Args&&... args) {
co_await SomeCondition();
std::invoke(f, std::forward<Args>(args)...);
}
which is the temporary object with value &Object::SomeMethod, which of course dies at the end of its full-expression, i.e., at the semicolon at the end of return WaitForConditionAndInvoke(&Object::SomeMethod, &object);. As soon as you hit that semicolon, all of the temporaries go out of scope and any references to them (like, that f captured in the coroutine frame) are definitely dangling.
Related
I have the following code which is used to call a function on an object and pass any argument in a perfect-forwarding way
template <typename F, typename T>
inline auto call_with_args(F&& f, T&& t) {
return [f = std::forward<F>(f), t = std::forward<T>(t)]
(auto&&... args) mutable { return (t.*f)(std::forward<decltype(args)>(args)...); };
}
I'm trying to read this but I'm unsure what [f = std::forward<F>(f)] does. That is the 'capture by value' syntax, but what does that have to do with std::forward (which is a cast to rvalue reference from what I could read)? Is it re-defining f with a new variable which gets assigned whatever type and value category f had after being universal-reference-bound to the function parameter f?
Can somebody explain this to me in simple terms please?
f and t are what as known as forwarding references and like all references, they are considered an lvalue, even if they refer to an rvalue.
So, since they are lvalues if you just captured them by value like
return [f, t](...) { ... };
Then f and t will be copied into the closure object as they are lvalues. This isn't great as we are missing out on being able to move f and t if they refer to rvalues . To avoid this we use std::forward which cast rvalues back to rvalues while leaving lvalues as lvalues.
That gives you the syntax of
return [f = std::forward<F>(f)](...) { ... };
which says create a member of the lambda named f and initialize it with std::forward<F>(f) where that f is the f from the surrounding scope.
Let's go through it one by one:
If an lvalue reference is passed in, f = ... will have an lvalue reference as the .... A copy is made.
If an rvalue reference is passed in, f = ... will have an rvalue as the .... This means that the value is moved into the lambda capture instead.
As a result, the code saves on making a copy when it's passed an rvalue reference.
There's another way that achieves the same thing and is potentially a little less confusing to the reader: If you're going to keep the object anyway, just take it by value, then move.
template <typename F, typename T>
inline auto call_with_args(F f, T t) {
return [f = std::move(f), t = std::move(t)]
(auto&&... args) mutable { return (t.*f)(std::forward<decltype(args)>(args)...); };
}
That's a bit easier to reason about and achieves the same thing: A copy when an lvalue is passed in, and no copy when an rvalue (temporary) is passed in. (It does use one extra move, so if your objects are potentially expensive to move like std::array, you might want to stick to the original version.)
Obviously it is possible to pass an rvalue reference to std::thread constructor. My problem is with definition of this constructor in cppreference. It says that this constructor:
template< class Function, class... Args >
explicit thread( Function&& f, Args&&... args );
Creates new std::thread object and associates it with a thread of
execution. First the constructor copies/moves all arguments (both the
function object f and all args...) to thread-accessible storage as if
by the function:
template <class T>
typename decay<T>::type decay_copy(T&& v) {
return std::forward<T>(v);
}
As far as I can check:
std::is_same<int, std::decay<int&&>::type>::value
returns true. This means std::decay<T>::type will drop rvalue reference part of argument. Then how std::thread constructor knows that which argument is passed by lvalue or rvalue references? Because all T& and T&& will be converted to T by std::decay<T>::type
The std::thread constructor knows the value category of its arguments, because it knows what Function and Args... are, which it uses to perfectly forward the its parameters to decay_copy (or equivalent).
The actual thread function doesn't know the value category. It's always invoked as an rvalue, with all rvalue arguments - which makes sense: the copies of f and args... are local to the thread, and won't be used anywhere else.
auto s = std::decay_copy(std::string("hello"));
Is equivalent to:
template<>
std::string std::decay_copy<std::string>(std::string&& src) {
return std::string(std::move(src));
}
std::string s = decay_copy<std::string>(std::string("hello"));
It is common problem of the perfect forwarding. If you want to restore information about rvalue in the function, you have to use std::forward std::forward . If you are interested in the value type detection you may read this value_category . From the description you can find the information how the compiler recognizes rvalue, xvalue, lvalue, prvalue, gvalue on compile time.
I wanted to have some kind of delegator class. Shortened version of my approach is below and it's main functionality is to start new thread doing some thing (in this example it prints text every second):
void Flusher::start(){
m_continue.store(true);
m_thread = std::thread([](std::atomic<bool>& shouldContinue){
while(shouldContinue.load()){
std::this_thread::sleep_for(std::chrono::seconds(1));
std::cout << "sec passed" << std::endl;
}}, std::ref<std::atomic<bool>>(m_continue)
);
}
My concern is, that std::thread constructor has following signature:
template< class Function, class... Args >
explicit thread( Function&& f, Args&&... args );
So it takes rvalue reference as the first and second argument. If it's so, then I should not use shouldContinue after passing it to the std::thread constructor as it was moved.
Of course I want to have control over this function and therefore I want to use shouldContinue in a caller thread to stop called function. For obvious reasons I do not want to make this variable global.
I think, that std::ref makes some magic there, but I am still not sure how does it work (I saw std::ref in some examples when creating new thread).
I tried to not care at all about the fact, this is rvalue reference and I used shouldContinue later on and nothing crashed, but I fear this is simply undefined behavior. Could anyone tell if above code is correct and if not, how to do this correctly?
There is a special type deduction rule when && is used with templates.
Check this out for a really good explanation:
http://eli.thegreenplace.net/2014/perfect-forwarding-and-universal-references-in-c/
template <class T>
void func(T&& t) {
}
"When && appears in a type-deducing context, T&& acquires a special meaning. When func is instantiated, T depends on whether the argument passed to func is an lvalue or an rvalue. If it's an lvalue of type U, T is deduced to U&. If it's an rvalue, T is deduced to U:"
func(4); // 4 is an rvalue: T deduced to int
double d = 3.14;
func(d); // d is an lvalue; T deduced to double&
float f() {...}
func(f()); // f() is an rvalue; T deduced to float
int bar(int i) {
func(i); // i is an lvalue; T deduced to int&
}
Also, reference collapsing rule is a good read.
I'm having trouble with std::functions created from lambdas if the function returns a reference but the return type isn't explicitly called out as a reference. It seems that the std::function is created fine with no warnings, but upon calling it, a value is returned when a reference is expected, causing things to blow up. Here's a very contrived example:
#include <iostream>
#include <vector>
#include <functional>
int main(){
std::vector<int> v;
v.push_back(123);
std::function<const std::vector<int>&(const std::vector<int>&)> callback =
[](const std::vector<int> &in){return in;};
std::cout << callback(v).at(0) << std::endl;
return 0;
}
This prints out garbage, however if the lambda is modified to explicitly return a const reference it works fine. I can understand the compiler thinking the lambda is return-by-value without the hint (when I originally ran into this problem, the lambda was directly returning the result from a function that returned a const reference, in which case I would think that the const reference return of the lambda would be deducible, but apparently not.) What I am surprised by is that the compiler lets the std::function be constructed from the lambda with mismatched return types. Is this behavior expected? Am I missing something in the standard that allows this mismatch to occur? I'm seeing this with g++ (GCC) 4.8.2, haven't tried it with anything else.
Thanks!
Why is it broken?
When the return type of a lambda is deduced, reference and cv-qualifications are dropped. So the return type of
[](const std::vector<int> &in){return in;};
is just std::vector<int>, not std::vector<int> const&. As a result, if we strip out the lambda and std::function part of your code, we effectively have:
std::vector<int> lambda(std::vector<int> const& in)
{
return in;
}
std::vector<int> const& callback(std::vector<int> const& in)
{
return lambda(in);
}
lambda returns a temporary. It effectively is just copied its input. This temporary is bound the reference return in callback. But temporaries bound to a reference in a return statement do not have their lifetime extended, so the temporary is destroyed at the end of the return statement. Thus, at this point:
callback(v).at(0)
-----------^
we have a dangling reference to a destroyed copy of v.
The solution is to explicitly specify the return type of the lambda to be a reference:
[](const std::vector<int> &in)-> const std::vector<int>& {return in;}
[](const std::vector<int> &in)-> decltype(auto) {return in;} // C++14
Now there are no copies, no temporaries, no dangling references, and no undefined behavior.
Who's at fault?
As to whether this is expected behavior, the answer is actually yes. The conditions for constructibility of a std::function are [func.wrap.func.con]:
f is Callable (20.9.12.2) for argument types ArgTypes... and return type R.
where, [func.wrap.func]:
A callable object f of type F is Callable for argument types ArgTypes and return type R if the expression
INVOKE (f, declval<ArgTypes>()..., R), considered as an unevaluated operand (Clause 5), is well
formed (20.9.2).
where, [func.require], emphasis mine:
Define INVOKE(f, t1, t2, ..., tN, R) as static_cast<void>(INVOKE (f, t1, t2, ..., tN)) if R is cv void, otherwise INVOKE(f, t1, t2, ..., tN) implicitly converted to R.
So, if we had:
T func();
std::function<T const&()> wrapped(func);
That actually meets all the standard requirements: INVOKE(func) is well-formed and while it returns T, T is implicitly convertible to T const&. So this isn't a gcc or clang bug. This is likely a standard defect, as I don't see why you would ever want to allow such a construction. This will never be valid, so the wording should likely require that if R is a reference type then F must return a reference type as well.
I did a bit of my own searching regarding the std::function constructor. It seems this part is an oversight in the interaction of std::function and the standard's Callable concept. std::function<R(Args...)>::function<F>(F) requires F to be Callable as R(Args...), which in itself seems reasonable. Callable for R(Args...) requires F's return type (when given arguments of types Args... to be implicitly convertible to R, which also in itself seems reasonable. Now when R is const R_ &, this will an allow implicit conversion of R_ to const R_ & because const references are allowed to bind to rvalues. This is not necessarily unsafe. E.g. consider a function f() that returns an int, but is considered callable as const int &().
const int &result = f();
if ( f == 5 )
{
// ...
}
There is no issue here because of C++'s rules for extending the lifetime of a temporary. However, the following has undefined behavior:
std::function<const int &()> fWrapped{f};
if ( fWrapped() == 5 )
{
// ...
}
This is because lifetime extension does not apply here. The temporary is created inside of std::function's operator() and is destroyed before the comparison.
Therefore, std::function's constructor probably should not rely on Callable alone, but enforce the additional restriction that implicit conversion of an rvalue to a const lvalue in order to bind to a reference is forbidden. Alternatively, Callable could be changed to never allow this conversion, at the expense of disallowing some safe usage (if only because of lifetime extension).
To make things more complicated yet, fWrapped() from the above example is perfectly safe to call, as long as you don't access the target of the dangling reference.
If you use:
return std::ref(in);
In your lambda it will work.
This will make your lambda's return type a std::reference_wrapper<std::vector<int>> which is implicitly convertible to std::vector<int>&.
I wish to create a wrapper around std::make_pair that takes a single argument and uses that argument to make the first and second members of the pair. Furthermore, I wish to take advantage of move semantics.
Naively, we might write (ignoring return types for clarity),
template <typename T>
void foo(T&& t)
{
std::make_pair(std::forward<T>(t),
std::forward<T>(t));
}
but this is unlikely to do what we want.
What we want is:
In the case where foo is called with a (const) lvalue reference argument, we should pass that (const) reference on to std::make_pair unmodified for both arguments.
In the case where foo is called with an rvalue reference argument, we should duplicate the referenced object, then call std::make_pair with the original rvalue reference as well as an rvalue reference to the newly created object.
What I've come up with so far is:
template <typename T>
T forward_or_duplicate(T t)
{
return t;
}
template <typename T>
void foo(T&& t)
{
std::make_pair(std::forward<T>(t),
forward_or_duplicate<T>(t));
}
But I'm reasonably sure it's wrong.
So, questions:
Does this work? I suspect not in that if foo() is called with an rvalue reference then T's move constructor (if it exists) will be called when constructing the T passed by value to forward_or_duplicate(), thus destroying t.
Even if it does work, is it optimal? Again, I suspect not in that T's copy constructor will be called when returning t from forward_or_duplicate().
This seems like a common problem. Is there an idiomatic solution?
So, questions:
Does this work? I suspect not in that if foo() is called with an rvalue reference then T's move constructor (if it exists) will be
called when constructing the T passed by value to
forward_or_duplicate(), thus destroying t.
No, t in foo is an lvalue, so constructing the T passed by value to
forward_or_duplicate() from t calls the copy constructor.
Even if it does work, is it optimal? Again, I suspect not in that T's copy constructor will be called when returning t from
forward_or_duplicate().
No, t is a function parameter, so the return implicitly moves, and doesn't copy.
That said, this version will be more efficient and safer:
template <typename T>
T forward_or_duplicate(std::remove_reference_t<T>& t)
{
return t;
}
If T is an lvalue reference, this results in the same signature as before. If T is not a reference, this saves you a move. Also, it puts T into a non-deduced context, so that you can't forget to specify it.
Your exact code works. Slight variations of it (ie, not calling make_pair but some other function) result in unspecified results. Even if it appears to work, subtle changes far from this line of code (which are locally correct) can break it.
Your solution isn't optimal, because it can copy a T twice, even when it works, when it only needs to copy it once.
This is by far the easiest solution. It doesn't fix the subtle breaks caused by code elsewhere changing, but if you are really calling make_pair that is not a concern:
template <typename T>
void foo(T&& t) {
std::make_pair(std::forward<T>(t),
static_cast<T>(t));
}
static_cast<T>(t) for a deduced type T&& is a noop if T&& is an lvalue, and a copy if T&& is an rvalue.
Of course, static_cast<T&&>(t) can also be used in place of std::forward<T>(t), but people don't do that either.
I often do this:
template <typename T>
void foo(T&& t) {
T t2 = t;
std::make_pair(std::forward<T>(t),
std::forward<T>(t2));
}
but that blocks a theoretical elision opportunity (which does not occur here).
In general, calling std::forward<T>(t) on the same function call as static_cast<T>(t) or any equivalent copy-or-forward function is a bad idea. The order in which arguments are evaluated is not specified, so if the argument consuming std::forward<T>(t) is not of type T&&, and its constructor sees the rvalue T and moves state out of it, the static_cast<T>(t) could evaluate after the state of t has been ripped out.
This does not happen here:
template <typename T>
void foo(T&& t) {
T t2 = t;
std::make_pair(std::forward<T>(t),
std::forward<T>(t2));
}
because we move the copy-or-forward to a different line, where we initialize t2.
While T t2=t; looks like it always copies, if T&& is an lvalue reference, T is also an lvalue reference, and int& t2 = t; doesn't copy.