I'm trying to make a function that accepts a shared pointer to some functor. With manually crafted functors there're no problems, but with lambda there are. I know that I can't use decltype with lambda - every new lambda declaration creates a new type. Right now I'm writing:
auto lambda = [](int a, float b)->int
{
return 42;
};
using LambdaType = decltype(lambda);
shared_ptr<LambdaType> ptr{ new LambdaType{ lambda } };
It works, but looks ugly. Moreover there's a copy constructor call! Is there any way to simplify?
You could use std::function as type.
Lambdas are merely auto written invokable objects to make simple code simple. It you want something beyond their default automatic storage behavior, write the type they write yourself.
It is illegal to have a lambda type in an unevaluated context. In an evaluated context, it creates a lambda in automatic storage. You want it on the free store. This requires at least logically a copy.
A horrible hack involving violating the unevaluated context rule, sizeof/alignof, aligned_storage_t, placement new, possibly unbounded compile time recursion (or maybe one with a static_assert), returning pointers to local variables, and the aliasing constructor of shared ptr, and requiring callers to write insane code might avoid calling the copy/move. But it is a bad idea, and simply using invokable objects is easier.
Of course, accepting the copy/move makes it trivial. But at that point, just use std::function unless you need something like varargs.
You state you do not want to force users to use std::function; but std::function would implicitly convert a compatible lambda into itself.
If you are willing to accept a copy, we can do this:
template<class T>
std::shared_ptr<std::decay_t<T>>
auto_shared( T&& t ) {
return std::make_shared<std::decay_t<T>>(std::forward<T>(t));
}
then auto ptr = auto_shared( [x=0]()mutable{ return x++; } ); is a non-type-erased shared pointer to a counting lambda. The lambda is copied (well, moved) into the shared storage.
If you want to avoid that copy, the client can write a manual function object and call make_shared<X>(ctor_args) on it.
There is no reasonable way to separate a lambdas type from its construction in C++ at this point.
if you catch something in lambda, it becomes algorithmically same as std::function, so use it freely. Also, std::function implements captured values memory management, so using std::shared_ptr on top of it is not required.
If you catch nothing, lambda is convertible to simple function pointer:
int(*ptr)(int,int) = [](int a, int b) -> int {
return a+b;
};
Functions are allocated statically and definitely shouldn't be deleted. So, you don't actually need std::shared_ptr
Related
The reason for me to ask this is I need to store std::function in a vector, and the in-house vector we have in company basically is doing realloc if it needs more memory. (Basically just memcpy, no copy/move operator involves)
This means all the element we can put in our container need to be trivially-copyable.
Here is some code to demonstrate the problematic copy I had:
void* func1Buffer = malloc(sizeof(std::function<void(int)>));
std::function<void(int)>* func1p = new (func1Buffer) std::function<void(int)>();
std::function<void(int)>* func2p = nullptr;
*func1p = [](int) {};
char func2Buffer[sizeof(*func1p)];
memcpy(&func2Buffer, func1p, sizeof(*func1p));
func2p = (std::function<void(int)>*)(func2Buffer);
// func2p is still valid here
(*func2p)(10);
free(func1Buffer);
// func2p is now invalid, even without std::function<void(int)> desctructor get triggered
(*func2p)(10);
I understand we should support copy/move of the element in order to store std::function safely.
But I am still very curious about what is the direct cause of invalid std::function copy above.
----------------------------------------------------UpdateLine----------------------------------------------------
Updated the code sample.
I have found the direct reason for this failure, by debugging our in-house vector more.
The trivially copied std::function has some dependency on original object memory, delete the original memory will trash the badly copied std::function even without the destruction of the original object.
Thanks for everyone's answer to this post. It's all valuable input. :)
The problem is how std::function has to be implemented: it has to manage the lifetime of whatever object it's holding onto. So when you write:
{
std::function<Sig> f = X{};
}
we must invoke the destructor of X when f goes out of scope. Moreover, std::function will [potentially] allocate memory to hold that X so the destructor of f must also [potentially] free that memory.
Now consider what happens when we try to do:
char buffer[100000]; // something big
{
std::function<void()> f = X{};
memcpy(buffer, &f, sizeof(f));
}
(*reinterpret_cast<std::function<void()>*>(buffer))();
At the point we're calling the function "stored" at buffer, the X object has already been destroyed and the memory holding it has been [potentially] freed. Regardless of whether X were TriviallyCopyable, we don't have an X anymore. We have the artist formerly known as an X.
Because it's incumbent upon std::function to manage its own objects, it cannot be TriviallyCopyable even if we added the requirement that all callables it managed were TriviallyCopyable.
To work in your realloc_vector, you need either need something like function_ref (or std::function<>*) (that is, a type that simply doesn't own any resources), or you need to implement your own version of function that (a) keeps its own storage as a member to avoid allocating memory and (b) is only constructible with TriviallyCopyable callables so that it itself becomes trivially copyable. Whichever solution is better depends on the what your program is actually doing.
But I am still very curious about what is the direct cause of invalid
std::function copy above.
std::function cannot be TriviallyCopyable (or conditionally TriviallyCopyable) because as a generic callable object wrapper it cannot assume that the stored callable is TriviallyCopyable.
Consider implementing your own version of std::function that only supports TriviallyCopyable callable objects (using a fixed buffer for storage), or use a vector of function pointers if applicable in your situation.
To be trivially copyable is something that is inherently related to a given type, not to an object.
Consider the following example:
#include<type_traits>
#include<functional>
int main() {
auto l = [](){};
static_assert(not std::is_trivially_copyable<decltype(l)>::value, "!");
std::function<void(void)> f;
bool copyable = std::is_trivially_copyable<decltype(f)>::value;
f = l;
// do something based on the
// fact that f is trivially copyable
}
How could you enforce the property once you have assigned to the function the lambda, that is not trivially copyable?
What you are looking for would be a runtime machinery that gets a decision based on the actual object assigned to the function.
This is not how std::is_trivially_copyable works.
Therefore the compiler has to make a decision at compile-time regarding the given specialization for the std::function. For it's a generic container for callable objects and you can assign it trivially copyable objects as well as objects that aren't trivially copyable, the rest goes without saying.
A std::function might allocate memory for captured variables. As with any other class which allocates memory, it's not trivially copyable.
I want to store and identify std::function objects in a std::map.
To identify I want to use the std::function::target.
I can't get the pointer from std::function::target if I use std::bind to bind to a member function from a class.
#include <functional>
#include <iostream>
using namespace std;
void normal_fun(int a)
{
std::cout << "hello\n"<<a;
}
class testclass
{
public:
int b;
void mytest(int a)
{
b = a;
}
};
uintptr_t getFuncAddress(std::function<void(int)> &f)
{
uintptr_t returnAddress = reinterpret_cast<uintptr_t>(f.target<void(*)(int)>());
return returnAddress;
}
int main()
{
testclass c;
std::function<void(int)> f1 = bind(&testclass::mytest,&c, placeholders::_1);
std::function<void(int)> f2 = normal_fun;
auto addrF1 = getFuncAddress(f1);
auto addrF2 = getFuncAddress(f2);
}
How can I achieve what I want to do?
A std::function is not a function pointer. It is not even a pointer.
If you know what type is stored in a std::function, you can get a pointer to what it stores. Note that the pointer here is not the function pointer, but a pointer to the function pointer.
If you do not know what type it stores, you cannot.
If you want < or == or hash (etc), std::function does not provide this for you. It type erases (), copy, move, destruction, typeid, and cast-back-to-original.
You can use type erasure techniques to augment a std::function with those operations; note that type erasure on binary operations is a touch trickier than type erasure is in general.
Type erasing something shouldn't be your first go-to when solving a problem, but it will solve your problem. There are articles on type erasure in SO documentation for C++. This isn't a beginner subject.
Odds are your underlying problem can be solved in a much simpler way.
In any case, using the type returned from target to order is not a great idea, as it is a pointer to a possibly automatic storage object, or possibly heap storage object; the two of which are going to have significantly different invalidation rules under innocuous operations (like move). A SBO (small buffer optimization) target is going to move with the instance of the std::function, while a heap-allocated one is likely to stay with the state the std::function moves around under std::move-like operations. This is a real mess.
The point of std::function is to give a uniform interface and type for callable objects that meet a given signature. You are expecting it to also provide a uniform key for sorting, but it doesn't do that. There is no way to sort function pointers and arbitrary function objects and callables returned by std::bind. They are completely different things, and comparing them doesn't make sense. All std::function allows you to do is store them and call them.
If you need a sorting mechanism you'll have to invent something yourself, you won't get it from std::function.
I want to store and identify std::function objects in a std::map.
I assume, you want to identify std::function objects (e.g. to call or delete them selectively).
In such situations, I use an additional key e.g. a simple unsigned.
A global function may be used:
typedef unsigned Key;
Key genId()
{
static Key id = 0;
return ++id;
}
Now, the std::function objects can be paired with this key. The API may grant that access to paired std::function objects can be done using the key exclusively.
f.target<void(*)(int)>
Is wrong, because the type returned by bind is not void(*)(int), so the target will never return a non-null pointer. Bind does not return a function pointer. It returns some object of implementation specified type.
The type can be acquired with decltype. This is a bit perverse, but should be correct:
f.target<decltype(bind(
&testclass::mytest,
(testclass*)nullptr,
placeholders::_1)
)>
but i have the Problem that i didn't know in getFuncAddress if i have testclass or myOtherClass or someone else Class
You can only use std::function::target if you know the type of the wrapped object. If the number of choices is limited, then you can try to get the target with each type until non-null is returned. For an arbitrary unknown type, std::function::target can not be of any use for you.
I want to store and identify std::function objects in a std::map
I think you will have to use some external key, that cannot be extracted from the std::function. This means that if you wish to prevent duplicates, you also need an external method of guaranteeing a unique mapping from key to the function.
The reason for me to ask this is I need to store std::function in a vector, and the in-house vector we have in company basically is doing realloc if it needs more memory. (Basically just memcpy, no copy/move operator involves)
This means all the element we can put in our container need to be trivially-copyable.
Here is some code to demonstrate the problematic copy I had:
void* func1Buffer = malloc(sizeof(std::function<void(int)>));
std::function<void(int)>* func1p = new (func1Buffer) std::function<void(int)>();
std::function<void(int)>* func2p = nullptr;
*func1p = [](int) {};
char func2Buffer[sizeof(*func1p)];
memcpy(&func2Buffer, func1p, sizeof(*func1p));
func2p = (std::function<void(int)>*)(func2Buffer);
// func2p is still valid here
(*func2p)(10);
free(func1Buffer);
// func2p is now invalid, even without std::function<void(int)> desctructor get triggered
(*func2p)(10);
I understand we should support copy/move of the element in order to store std::function safely.
But I am still very curious about what is the direct cause of invalid std::function copy above.
----------------------------------------------------UpdateLine----------------------------------------------------
Updated the code sample.
I have found the direct reason for this failure, by debugging our in-house vector more.
The trivially copied std::function has some dependency on original object memory, delete the original memory will trash the badly copied std::function even without the destruction of the original object.
Thanks for everyone's answer to this post. It's all valuable input. :)
The problem is how std::function has to be implemented: it has to manage the lifetime of whatever object it's holding onto. So when you write:
{
std::function<Sig> f = X{};
}
we must invoke the destructor of X when f goes out of scope. Moreover, std::function will [potentially] allocate memory to hold that X so the destructor of f must also [potentially] free that memory.
Now consider what happens when we try to do:
char buffer[100000]; // something big
{
std::function<void()> f = X{};
memcpy(buffer, &f, sizeof(f));
}
(*reinterpret_cast<std::function<void()>*>(buffer))();
At the point we're calling the function "stored" at buffer, the X object has already been destroyed and the memory holding it has been [potentially] freed. Regardless of whether X were TriviallyCopyable, we don't have an X anymore. We have the artist formerly known as an X.
Because it's incumbent upon std::function to manage its own objects, it cannot be TriviallyCopyable even if we added the requirement that all callables it managed were TriviallyCopyable.
To work in your realloc_vector, you need either need something like function_ref (or std::function<>*) (that is, a type that simply doesn't own any resources), or you need to implement your own version of function that (a) keeps its own storage as a member to avoid allocating memory and (b) is only constructible with TriviallyCopyable callables so that it itself becomes trivially copyable. Whichever solution is better depends on the what your program is actually doing.
But I am still very curious about what is the direct cause of invalid
std::function copy above.
std::function cannot be TriviallyCopyable (or conditionally TriviallyCopyable) because as a generic callable object wrapper it cannot assume that the stored callable is TriviallyCopyable.
Consider implementing your own version of std::function that only supports TriviallyCopyable callable objects (using a fixed buffer for storage), or use a vector of function pointers if applicable in your situation.
To be trivially copyable is something that is inherently related to a given type, not to an object.
Consider the following example:
#include<type_traits>
#include<functional>
int main() {
auto l = [](){};
static_assert(not std::is_trivially_copyable<decltype(l)>::value, "!");
std::function<void(void)> f;
bool copyable = std::is_trivially_copyable<decltype(f)>::value;
f = l;
// do something based on the
// fact that f is trivially copyable
}
How could you enforce the property once you have assigned to the function the lambda, that is not trivially copyable?
What you are looking for would be a runtime machinery that gets a decision based on the actual object assigned to the function.
This is not how std::is_trivially_copyable works.
Therefore the compiler has to make a decision at compile-time regarding the given specialization for the std::function. For it's a generic container for callable objects and you can assign it trivially copyable objects as well as objects that aren't trivially copyable, the rest goes without saying.
A std::function might allocate memory for captured variables. As with any other class which allocates memory, it's not trivially copyable.
Let's say I have a function which takes an std::function:
void callFunction(std::function<void()> x)
{
x();
}
Should I pass x by const-reference instead?:
void callFunction(const std::function<void()>& x)
{
x();
}
Does the answer to this question change depending on what the function does with it? For example if it is a class member function or constructor which stores or initializes the std::function into a member variable.
If you want performance, pass by value if you are storing it.
Suppose you have a function called "run this in the UI thread".
std::future<void> run_in_ui_thread( std::function<void()> )
which runs some code in the "ui" thread, then signals the future when done. (Useful in UI frameworks where the UI thread is where you are supposed to mess with UI elements)
We have two signatures we are considering:
std::future<void> run_in_ui_thread( std::function<void()> ) // (A)
std::future<void> run_in_ui_thread( std::function<void()> const& ) // (B)
Now, we are likely to use these as follows:
run_in_ui_thread( [=]{
// code goes here
} ).wait();
which will create an anonymous closure (a lambda), construct a std::function out of it, pass it to the run_in_ui_thread function, then wait for it to finish running in the main thread.
In case (A), the std::function is directly constructed from our lambda, which is then used within the run_in_ui_thread. The lambda is moved into the std::function, so any movable state is efficiently carried into it.
In the second case, a temporary std::function is created, the lambda is moved into it, then that temporary std::function is used by reference within the run_in_ui_thread.
So far, so good -- the two of them perform identically. Except the run_in_ui_thread is going to make a copy of its function argument to send to the ui thread to execute! (it will return before it is done with it, so it cannot just use a reference to it). For case (A), we simply move the std::function into its long-term storage. In case (B), we are forced to copy the std::function.
That store makes passing by value more optimal. If there is any possibility you are storing a copy of the std::function, pass by value. Otherwise, either way is roughly equivalent: the only downside to by-value is if you are taking the same bulky std::function and having one sub method after another use it. Barring that, a move will be as efficient as a const&.
Now, there are some other differences between the two that mostly kick in if we have persistent state within the std::function.
Assume that the std::function stores some object with a operator() const, but it also has some mutable data members which it modifies (how rude!).
In the std::function<> const& case, the mutable data members modified will propagate out of the function call. In the std::function<> case, they won't.
This is a relatively strange corner case.
You want to treat std::function like you would any other possibly heavy-weight, cheaply movable type. Moving is cheap, copying can be expensive.
If you're worried about performance, and you aren't defining a virtual member function, then you most likely should not be using std::function at all.
Making the functor type a template parameter permits greater optimization than std::function, including inlining the functor logic. The effect of these optimizations is likely to greatly outweigh the copy-vs-indirection concerns about how to pass std::function.
Faster:
template<typename Functor>
void callFunction(Functor&& x)
{
x();
}
As usual in C++11, passing by value/reference/const-reference depends on what you do with your argument. std::function is no different.
Passing by value allows you to move the argument into a variable (typically a member variable of a class):
struct Foo {
Foo(Object o) : m_o(std::move(o)) {}
Object m_o;
};
When you know your function will move its argument, this is the best solution, this way your users can control how they call your function:
Foo f1{Object()}; // move the temporary, followed by a move in the constructor
Foo f2{some_object}; // copy the object, followed by a move in the constructor
Foo f3{std::move(some_object)}; // move the object, followed by a move in the constructor
I believe you already know the semantics of (non)const-references so I won't belabor the point. If you need me to add more explanations about this, just ask and I'll update.
This is probably a philosophical question, but I ran into the following problem:
If you define an std::function, and you don't initialize it correctly, your application will crash, like this:
typedef std::function<void(void)> MyFunctionType;
MyFunctionType myFunction;
myFunction();
If the function is passed as an argument, like this:
void DoSomething (MyFunctionType myFunction)
{
myFunction();
}
Then, of course, it also crashes. This means that I am forced to add checking code like this:
void DoSomething (MyFunctionType myFunction)
{
if (!myFunction) return;
myFunction();
}
Requiring these checks gives me a flash-back to the old C days, where you also had to check all pointer arguments explicitly:
void DoSomething (Car *car, Person *person)
{
if (!car) return; // In real applications, this would be an assert of course
if (!person) return; // In real applications, this would be an assert of course
...
}
Luckily, we can use references in C++, which prevents me from writing these checks (assuming that the caller didn't pass the contents of a nullptr to the function:
void DoSomething (Car &car, Person &person)
{
// I can assume that car and person are valid
}
So, why do std::function instances have a default constructor? Without default constructor you wouldn't have to add checks, just like for other, normal arguments of a function.
And in those 'rare' cases where you want to pass an 'optional' std::function, you can still pass a pointer to it (or use boost::optional).
True, but this is also true for other types. E.g. if I want my class to have an optional Person, then I make my data member a Person-pointer. Why not do the same for std::functions? What is so special about std::function that it can have an 'invalid' state?
It does not have an "invalid" state. It is no more invalid than this:
std::vector<int> aVector;
aVector[0] = 5;
What you have is an empty function, just like aVector is an empty vector. The object is in a very well-defined state: the state of not having data.
Now, let's consider your "pointer to function" suggestion:
void CallbackRegistrar(..., std::function<void()> *pFunc);
How do you have to call that? Well, here's one thing you cannot do:
void CallbackFunc();
CallbackRegistrar(..., CallbackFunc);
That's not allowed because CallbackFunc is a function, while the parameter type is a std::function<void()>*. Those two are not convertible, so the compiler will complain. So in order to do the call, you have to do this:
void CallbackFunc();
CallbackRegistrar(..., new std::function<void()>(CallbackFunc));
You have just introduced new into the picture. You have allocated a resource; who is going to be responsible for it? CallbackRegistrar? Obviously, you might want to use some kind of smart pointer, so you clutter the interface even more with:
void CallbackRegistrar(..., std::shared_ptr<std::function<void()>> pFunc);
That's a lot of API annoyance and cruft, just to pass a function around. The simplest way to avoid this is to allow std::function to be empty. Just like we allow std::vector to be empty. Just like we allow std::string to be empty. Just like we allow std::shared_ptr to be empty. And so on.
To put it simply: std::function contains a function. It is a holder for a callable type. Therefore, there is the possibility that it contains no callable type.
Actually, your application should not crash.
ยง 20.8.11.1 Class bad_function_call [func.wrap.badcall]
1/ An exception of type bad_function_call is thrown by function::operator() (20.8.11.2.4) when the function wrapper object has no target.
The behavior is perfectly specified.
One of the most common use cases for std::function is to register callbacks, to be called when certain conditions are met. Allowing for uninitialized instances makes it possible to register callbacks only when needed, otherwise you would be forced to always pass at least some sort of no-op function.
The answer is probably historical: std::function is meant as a replacement for function pointers, and function pointers had the capability to be NULL. So, when you want to offer easy compatibility to function pointers, you need to offer an invalid state.
The identifiable invalid state is not really necessary since, as you mentioned, boost::optional does that job just fine. So I'd say that std::function's are just there for the sake of history.
There are cases where you cannot initialize everything at construction (for example, when a parameter depends on the effect on another construction that in turn depends on the effect on the first ...).
In this cases, you have necessarily to break the loop, admitting an identifiable invalid state to be corrected later.
So you construct the first as "null", construct the second element, and reassign the first.
You can, actually, avoid checks, if -where a function is used- you grant that inside the constructor of the object that embeds it, you will always return after a valid reassignment.
In the same way that you can add a nullstate to a functor type that doesn't have one, you can wrap a functor with a class that does not admit a nullstate. The former requires adding state, the latter does not require new state (only a restriction). Thus, while i don't know the rationale of the std::function design, it supports the most lean & mean usage, no matter what you want.
Cheers & hth.,
You just use std::function for callbacks, you can use a simple template helper function that forwards its arguments to the handler if it is not empty:
template <typename Callback, typename... Ts>
void SendNotification(const Callback & callback, Ts&&... vs)
{
if (callback)
{
callback(std::forward<Ts>(vs)...);
}
}
And use it in the following way:
std::function<void(int, double>> myHandler;
...
SendNotification(myHandler, 42, 3.15);