In this question I talked about how to create a multi-function memoizator. In one of the answers, it was suggested to implement perfect forwarding, so passing the arguments to desired function will result inexpensive, like this:
template <typename ReturnType, typename... Args>
ReturnType callFunction(MemFunc<ReturnType, Args...> memFunc, Args&& ... args) //rvalue reference of args
{
auto it = multiCache.find(memFunc.name);
if (it == multiCache.end())
throw KeyNotFound(memFunc.name);
boost::any anyCachedFunc = it->second;
function < ReturnType(Args...)> cachedFunc = boost::any_cast<function<ReturnType(Args...)>> (anyCachedFunc);
return cachedFunc(forward<Args> (args) ...);//perfect forwarding
}
I'm not an expert of move semantic and perfect forwarding, but from what I've understood we need to pass an rvalue when we call callFunction.
So the first callFunction calling is illegal, while the second one is:
typedef vector<double> vecD;
MultiMemoizator mem;
//lambda function that returns the sorted vector (just for fun)
function<vecD(vecD)> sort_vec = [](vecD vec) {
sort(vec.begin(),vec.end());
return vec;
};
vector<vecD> vec;
// here fill vec...
mem.callFunction<double, double>(sortFunc, vec); //error! vec is an lvalue
mem.callFunction<double, double>(sortFunc, move(vec));//OK: move(vec) return an rvalue of the lvalue (vec)
The problem is that in this application, callFunction is supposed to be called several times on the same object, which is not supposed to happen once we call move on our object.
So for example:
mem.callFunction<double, double>(sortFunc, move(vec));//First time: sorting will take A LONG time because vec is not memoized
//now is not safe to use vec anymore...but it's memoized for sortFunc
mem.callFunction<double, double>(sortFunc, move(vec));//computed quickly because of memoization, but unsafe because of first move!
So from what I can understand I have to make a choice:
Give up on perfect forwarding implementing callFunction(MemFunc<ReturnType, Args...> memFunc, Args ... args) (or Args & ... args) , but introducing an expensive copy when we call cachedFunc(args)
Keep the efficient perfect forwarding, but introduce possible unexpected behaviour on multiple move on the same object
Obviously the second choice is not acceptable, so how can I avoid the first one?
A POSSIBLE SOLUTION:
The solution posted above is too much restrictive for the user: what if he wants, for some reason, to use vec again? So a possible solution could be to overload callFunction so it accept both solutions:
//rvalue reference/move/perfect forwarding implementation
template <typename ReturnType, typename... Args>
ReturnType callFunction(MemFunc<ReturnType, Args...> memFunc, Args&& ... args) {
...
return cachedFunc(forward<Args> (args) ...);//perfect forwarding
}
//lvalue reference/no perfect forwarding implementation
template <typename ReturnType, typename... Args>
ReturnType callFunction(MemFunc<ReturnType, Args...> memFunc, Args& ... args) {
...
return cachedFunc(args);//expensive!
}
This solution has some disadvantage or could lead to some problem?
Related
I am trying to figure out if my use of std::forward in the following code makes sense even though it is not a forwarding (=universal) reference. Please excuse the amount of code, but this is the stripped-down version of what I am trying to achieve.
template <class... Args>
class event_dispatcher {
private:
using func_t = std::function<bool(Args...)>;
public:
bool operator()(Args... args) const {
for (auto& f : funcs) {
if (!f(args...))
return false;
}
return true;
}
template <class F>
std::enable_if_t<std::is_invocable_r_v<bool, F, Args...>>
add(F&& f) {
funcs.emplace_back(std::forward<F>(f));
}
template <class F>
std::enable_if_t<!std::is_invocable_r_v<bool, F, Args...> && std::is_invocable_v<F, Args...>>
add(F&& f) {
add([f_ = std::forward<F>(f)](Args... args){
std::invoke(f_, std::forward<Args>(args)...); // <-- this is the one I am asking about!
return true;
});
}
private:
std::vector<func_t> funcs;
};
The idea is that if the callable passed to add() doesn't return a bool, we will wrap it in a lambda that does.
Now, if I just pass args... directly, it will always work, but if any of Args are rvalues then it will needlessly do a copy instead of a move. If I instead use std::move(args)... it will not work if any of Args is an lvalue. Using std::forward<Args>(args)... here seems to solve these problems and work as efficiently as possible in any case, but I am worried that I am missing something since I am using std::forward for a non-forwarding reference, and in general I am having a lot of trouble wrapping my head around the whole reference collapsing / rvalue reference / std::move / std::forward issues.
std::move does not move std::forward does not forward.
std::forward is a conditional move. std::forward<T> moves if T is a value or rvalue reference.
This lines up with when you want to move args..., so it is appropriate here.
A comment along those lines should be a good idea, as in any situation where you use std::forward outside of simple forwarding references.
So I'm trying to create a class which has a container for functors of a different type.
This is a simplified version of it.
template<class T>
class Container
{
public:
template<typename F, typename ... ARGS>
void addTask(F && func, ARGS && ... args);
private:
std::deque<std::function<T()>> container;
//.....
};
template<class T>
template<typename F, typename ... ARGS>
T Container<T>::addTask(F && func, ARGS && ... args);
{
container.emplace_back(std::bind(f,args...));
//.....
}
There are still few problems that I cannot solve yet.
Is there a way to remove std::bind and store a different object or a pointer?
Could this be more generic? Can I somehow store functions, which return different objects, in a single container(int,void...)?
Can some of the logic for creating the tasks be executed in compile time?Something like consexpr bind.
From a comment by the OP.
There are. This is simplified. I'm using futures and a special container in the real code. It is meant to be used in a multithreading environment
This is called burying the lede.
If you are storing callables to be invoked in other threads, in the other thread you want signature void(). In this thread you want a std::future to be populated.
As for binding arguments, while a number of std functions do this for you, I find it is best to ask for callables with pre-bound arguments. They can do it outside, using std::bind or lambdas or whatever other means they choose.
So this then comes
template<class Func,
class R = std::decay_t<std::result_of_t<Func const&()>>
>
std::future< R >
addTask( Func&& func ) {
auto task = std::packaged_task<R()>(std::forward<Func>(func));
auto ret = task.get_future();
container.push_back( std::packaged_task<void()>( std::move(task) ) );
return ret;
}
std::deque< std::packaged_task<void()> > container;
throw in some mutexes and shake and bake.
Here I use std::packaged_task<void()> as a pre-written move-only type-erased container for anything with that signature. We don't use the future it can produce, which is a waste, but it is shorter than writing your own move-only invoke-once owning function object.
I personally just wrote myself a light weight move-only std::function<void()> esque class instead of using std::packaged_task<void()>, but it was probably unwise.
The future returned from addTask gets fullfilled when the packaged_task<R()> is invoked, which is invoked when the packaged_task<void()> is invoked (possibly in another thread).
Outside of the structure, callers can give you any zero-argument callable object.
99 times out of 100, a simple [some_arg]{ some_code; } or even []{ some_code; } works. In complex cases they can mess around with std::bind or C++14 improvements with more complex lambdas.
Putting the storing of the arguments into addTask mixes the responsibility of the thread-task-queue with messing with arguments.
In fact, I'd write a thread-safe queue separately from my thread-pool, and have the thread-pool use it:
template<class T>
struct thread_safe_queue;
struct thread_pool {
thread_safe_queue< std::packaged_task<void()> > queue;
// etc
};
In C++17, a replacement for your bind looks like:
[
func = std::forward<Func>(func),
args = std::make_tuple( std::forward<Args>(args)... )
]() mutable {
std::apply( func, std::move(args) );
}
In C++14 you can write notstd::apply pretty easy. Move-into-lambda requires C++14, so if you need to efficiently move arguments you need std bind or a manual function object in C++11.
I will argue that placing the argument binding strongly in the domain of the code using the thread pool is best.
That also permits the thread pool to do things like pass the tasks optional extra arguments, like "cancellation tokens" or the like.
std::bind came from boost::bind, which was necessary before we had lambdas.
Unfortunately std::bind made it into the standard at the same time as lambdas, so it was immediately almost irrelevant.
In c++14 and beyond you can capture the function and args in a variadic lambda:
template<class T>
template<typename F, typename ... ARGS>
T Container<T>::addTask(F && func, ARGS && ... args)
{
container.emplace_back( [func = std::forward<F>(func),
args...]
() mutable // make mutable if you want to move the args in to func
{
return func(std::move(args)...);
});
//.....
}
You don't quite get perfect forwarding this way. There is a copy implicit in the capture of args...
This solved in c++17
template<class T>
template<typename F, typename ... ARGS>
T Container<T>::addTask(F && func, ARGS && ... args)
{
container.emplace_back( [func = std::forward<F>(func),
args = std::make_tuple(std::forward<ARGS>(args)...) ]
() mutable // make mutable if you want to move the args in to func
{
return std::apply(func, std::move(args));
});
//.....
}
Let's say I use std::forward_as_tuple to store the arguments of a function call in a tuple
auto args = std::forward_as_tuple(std::forward<Args>(args)...);
And then I pass this tuple by lvalue reference to a function that wants to invoke a function foo() with some of the arguments in args as determined by another std::integer_sequence. I do it with std::move() like so
template <typename TupleArgs, std::size_t... Indices>
decltype(auto) forward_to_foo(TupleArgs&& args,
std::index_sequence<Indices...>) {
return foo(std::get<Indices>(std::move(args))...);
}
And this would work because the rvalue qualified version of std::get<std::tuple> return std::tuple_element_t<Index, tuple<Types...>>&& which is an identity transformation of the reference-ness of the std::tuple_element_t<Index, tuple<Types...>> because of reference collapsing with the &&.
So if std::tuple_element_t<Index, tuple<Types...>> evaluates to T& the returned type would be T& && which is just T&. Similar reason for when std::tuple_element_t<Index, tuple<Types...>> returns T&& and T
Am I missing something? Are there some cases where this would fail?
template <typename TupleArgs, std::size_t... Indices>
decltype(auto) forward_to_foo(TupleArgs&& args,
std::index_sequence<Indices...>) {
return foo(std::get<Indices>(std::forward<TupleArgs>(args))...);
}
This is the correct implementation.
Use should look like:
auto tuple_args = std::forward_as_tuple(std::forward<Args>(args)...);
forward_to_foo( std::move(tuple_args), std::make_index_sequence<sizeof...(args)>{} );
there are a few differences here.
First, we take by forwarding reference, not by lvalue reference. This lets the caller provide rvalue (prvalue or xvalue) tuples to us.
Second, we forward the tuple into the std::get call. This means we only pass get an rvalue reference if the tuple was moved into us.
Third, we move into forward_to_foo. This ensures the above does the right thing.
Now, imagine if we wanted to call foo twice.
auto tuple_args = std::forward_as_tuple(std::forward<Args>(args)...);
auto indexes = std::make_index_sequence<sizeof...(args)>{};
forward_to_foo( tuple_args, indexes );
forward_to_foo( std::move(tuple_args), indexes );
we don't have to touch forward_to_foo at all, and we never move from any of the args more than once.
With your original implementation, any calls to forward_to_foo silently move from TupleArgs rvalue references or values without any indication at the call-site that we are destructive on the first parameter.
Other than that detail, yes that emulates forwarding.
Myself I'd just write a notstd::apply:
namespace notstd {
namespace details {
template <class F, class TupleArgs, std::size_t... Indices>
decltype(auto) apply(F&& f, TupleArgs&& args,
std::index_sequence<Indices...>) {
return std::forward<F>(f)(std::get<Indices>(std::forward<TupleArgs>(args))...);
}
}
template <class F, class TupleArgs>
decltype(auto) apply(F&& f, TupleArgs&& args) {
constexpr auto count = std::tuple_size< std::decay_t<TupleArgs> >{};
return details::apply(
std::forward<F>(f),
std::forward<TupleArgs>(args),
std::make_index_sequence< count >{}
);
}
}
then we do:
auto tuple_args = std::forward_as_tuple(std::forward<Args>(args)...);
auto call_foo = [](auto&&...args)->decltype(auto){ return foo(decltype(args)(args)...); };
return notstd::apply( call_foo, std::move(tuple_args) );
which moves the tricky bit into notstd::apply, which attempts to match the semantics of std::apply, which lets you replace it down the road with a more standard bit of code.
Can generic lambdas take advantage of the "Substitution Failure Is Not An Error" rule ? Example
auto gL =
[](auto&& func, auto&& param1, auto&&... params)
-> enable_if_t< is_integral<
std::decay_t<decltype(param1)>
>::value>
{
// ...
};
auto gL =
[](auto&& func, auto&& param1, auto&&... params)
-> enable_if_t< !is_integral<
std::decay_t<decltype(param1)>
>::value>
{
// ...
};
Are there any workarounds or plans to include this in the language ? Also since generic lambdas are templated function objects under the hood isn't it a bit odd that this can't be done ?
Lambdas are function objects under the hood. Generic lambdas are function objects with template operator()s.
template<class...Fs>
struct funcs_t{};
template<class F0, class...Fs>
struct funcs_t<F0, Fs...>: F0, funcs_t<Fs...> {
funcs_t(F0 f0, Fs... fs):
F0(std::move(f0)),
funcs_t<Fs...>(std::move(fs)...)
{}
using F0::operator();
using funcs_t<Fs...>::operator();
};
template<class F>
struct funcs_t<F>:F {
funcs_t(F f):F(std::move(f)){};
using F::operator();
};
template<class...Fs>
funcs_t< std::decay_t<Fs>... > funcs(Fs&&...fs) {
return {std::forward<Fs>(fs)...};
}
auto f_all = funcs( f1, f2 ) generates an object that is an overload of both f1 and f2.
auto g_integral =
[](auto&& func, auto&& param1, auto&&... params)
-> std::enable_if_t< std::is_integral<
std::decay_t<decltype(param1)>
>{}>
{
// ...
};
auto g_not_integral =
[](auto&& func, auto&& param1, auto&&... params)
-> std::enable_if_t< !std::is_integral<
std::decay_t<decltype(param1)>
>{}>
{
// ...
};
auto gL = funcs( g_not_integral, g_integral );
and calling gL will do SFINAE friendly overload resolution on the two lambdas.
The above does some spurious moves, which could be avoided, in the linear inheritance of funcs_t. In an industrial quality library, I might make the inheritance binary rather than linear (to limit instantiation depth of templates, and the depth of the inheritance tree).
As an aside, there are 4 reasons I know of to SFINAE enable lambdas.
First, with new std::function, you can overload a function on multiple different callback signatures.
Second, the above trick.
Third, currying a function object where it evaluates when it has the right number and type of args.
Forth, automatic tuple unpacking and similar. If I'm using continuation passing style, I can ask the passed in continuation if it will accept the tuple unpacked, or the future unbundled, etc.
A generic lambda can only have one body, so SFINAE wouldn't be of much use here.
One solution would be to package the call into a class which can store the result and is specialized on a void return type, encapsulating the void special handling away from your lambda. With a very little overhead, you can do this using the thread library facilities:
auto gL =
[](auto&& func, auto&&... params)
{
// start a timer
using Ret = decltype(std::forward<decltype(func)>(func)(
std::forward<decltype(params)>(params)...));
std::packaged_task<Ret()> task{[&]{
return std::forward<decltype(func)>(func)(
std::forward<decltype(params)>(params)...); }};
auto fut = task.get_future();
task();
// stop timer and print elapsed time
return fut.get();
};
If you want to avoid the overhead of packaged_task and future, it's easy to write your own version:
template<class T>
struct Result
{
template<class F, class... A> Result(F&& f, A&&... args)
: t{std::forward<F>(f)(std::forward<A>(args)...)} {}
T t;
T&& get() { return std::move(t); }
};
template<>
struct Result<void>
{
template<class F, class... A> Result(F&& f, A&&... args)
{ std::forward<F>(f)(std::forward<A>(args)...); }
void get() {}
};
auto gL =
[](auto&& func, auto&&... params)
{
// start a timer
using Ret = decltype(std::forward<decltype(func)>(func)(
std::forward<decltype(params)>(params)...));
Result<Ret> k{std::forward<decltype(func)>(func),
std::forward<decltype(params)>(params)...};
// stop timer and print elapsed time
return k.get();
};
The use of SFINAE is to remove an overload or a specialization from the candidate set when resolving a given function or template. In your case, we have a lambda - that is a functor with a single operator(). There is no overload, so there is no reason to use SFINAE1. The fact that the lambda is generic, which makes its operator() a function template, doesn't change that fact.
However, you don't actually need to differentiate between different return types. If func returns void for the given arguments, you can still return it. You just can't assign it to a temporary. But you don't have to do that either:
auto time_func = [](auto&& func, auto&&... params) {
RaiiTimer t;
return std::forward<decltype(func)>(func)(
std::forward<decltype(params)>(params)...);
};
Just write an RaiiTimer whose constructor starts a timer and whose destructor stops it and prints the result. This will work regardless of func's return type.
If you need something more complicated than that, then this is one of those cases where you should prefer a functor over a lambda.
1Actually, as Yakk points out, SFINAE could still be quite handy to check if your function is callable period, which isn't the problem you're trying to solve - so in this case, still not very helpful.
I'm trying hard for some hours and didn't manage to get this working.
I have a templated class spinlock:
template<typename T> class spinlock {
// ...
volatile T *shared_memory;
};
I'm trying to create something like this:
// inside spinlock class
template<typename F, typename... Ars>
std::result_of(F(Args...))
exec(F fun, Args&&... args) {
// locks the memory and then executes fun(args...)
};
But I'm trying to use a polymorphic function so that I can do this:
spinlock<int> spin;
int a = spin.exec([]() {
return 10;
});
int b = spin.exec([](int x) {
return x;
}, 10); // argument here, passed as x
// If the signature matches the given arguments to exec() plus
// the shared variable, call it
int c = spin.exec([](volatile int &shared) {
return shared;
}); // no extra arguments, shared becomes the
// variable inside the spinlock class, I need to make
// a function call that matches this as well
// Same thing, matching the signature
int d = spin.exec([](volatile int &shared, int x) {
return shared + x;
}, 10); // extra argument, passed as x... should match too
// Here, there would be an error
int d = spin.exec([](volatile int &shared, int x) {
return shared + x;
}); // since no extra argument was given
Basically, I'm trying to make an exec function that accepts F(Args...) or F(volatile T &, Args...) as an argument.
But I can't manage to make automatic detection of types.
How could I accomplish that?
Firstly, this signature will not compile:
// inside spinlock class
template<typename F, typename... Ars>
std::result_of(F(Args...))
exec(F fun, Args&&... args) {
// locks the memory and then executes fun(args...)
};
The return type needs to be
typename std::result_of<F(Args...)>::type
If your compiler implements N3436 then this function will not participate in overload resolution when fun(args...) is not a valid expression, but that is not required in C++11 and not implemented by many compilers yet. You will need to implement your own SFINAE check to prevent result_of giving an error when fun(args...) is not valid, or rewrite it without result_of
template<typename F, typename... Args>
auto
exec(F fun, Args&&... args) -> decltype(fun(std::forward<Args>(args)...))
{
// locks the memory and then executes fun(args...)
}
Then you can overload it for functions that need the additional parameter passed in:
template<typename F, typename... Args>
auto
exec(F fun, Args&&... args) -> decltype(fun(*this->shared_memory, std::forward<Args>(args)...))
{
// locks the memory and then executes fun(*shared_memory, args...)
}
When fun(std::forward<Args>(args)...) is not valid the first overload will not participate in overload resolution. When fun(*this->shared_memory, std::forward<Args>(args)...) is not valid the second overload will not participate in overload resolution. If neither is valid the call will be ill-formed, if both are valid the call will be ambiguous.