I have some code where I need to create unique-ids in some function to track
some objects manipulated by that function. I thought about using the address of a callback which the function expects.
The callbacks passed to the function are anonymous lambdas.
I call this function in many different places and, in every place, each lambda maintains its address across calls.
That is, even though the lambdas are temporary, each individual lambda has its same address every time it's passed to the function.
This is nice, for my use case. But, why do they have the same address, and not a new one every time? Most importantly, is this reliable across compilers?
Compiler: MSVC 2019
// A method within some class.
void Method() {
// Helper lambda
auto makeSignal = [this](State& state, Data& data, int foo, auto&& onClick) {
DoMakeSignal(state, data, true, foo, 3.14, std::forward<decltype(onClick)>(onClick));
};
// The unique-id will be the address of the lambda.
// Everytime this 'Method' is called, this lambda
// will have the same address. Nice, but why?
makeSignal(GetState(), GetData(), 5, [this] {
log("signal callback");
});
makeSignal(GetState(), GetData(), 42, [this] {
log("signal callback");
});
}
// In another file
void DoMakeSignal(State& state, Data&, bool, int, float, auto&& onClick) {
uintptr_t uid = reinterpret_cast<uintptr_t>(&onClick);
state.mLastObjectUid = uid;
// ...
}
That's by pure chance. The lambda object lives only until the end of the makeSignal call and after that the address which was occupied by the object can be reused.
There is no guarantee on whether a new lambda object (or any other object) will have the same address or not.
In particular that also means that after the call to makeSignal, if onClick was stored by-pointer or by-reference, calling it will result in UB, since the object is not alive anymore.
However, each lambda expression has its own type and its own call operator function, both of which are properties that could be used to obtain a unique id to identify the specific lambda expression.
If you want unique ids of lambda expressions, and only lambda expressions, then for example the following should work:
struct id_t {};
template<typename> id_t id_var;
template<typename T> constexpr auto id = &id_var<std::remove_cvref_t<T>>;
//...
state.mLastObjectUid = id<decltype(onClick)>; // type: id_t*
Related
I am trying to interface with a library written in c, that uses this familiar pattern:
void some_c_handler(void(*func)(void*), void* data);
Now, I want to write a C++ wrapper for this function that looks like this:
void my_new_cpp_handler(std::function<void()>&& func)
{
void (*p)() = foo(func);
void* data = bar(func);
some_c_handler(p, data);
}
Both some_c_handler and my_new_cpp_handler are solving the same problem; they're taking in some kind of function along with some state. But the latter is preferred in that it abstracts much of the implementation details from the user, and allows for simply passing in a lambda object.
So my_new_cpp_handler should take the func parameter it is given, convert it to a function pointer and pass its state on to data.
I don't know enough about the standard, or the implementation of std::function to know if this is even a reasonable request. Can foo and bar exist?
To put it differently, what I want is to be able to pass a stateful function to a c callback handler without having to manually instantiate my own struct type to pass along with it. Obviously std::function has already done this for us, so I'd love to be able to separate the function pointer from the state somehow and pass it onto the c-style handler. Is this possible?
Is this possible?
No.
You can wrap a C-style callback into an std::function<>..., and the compiler will emit code to extract the data and the handler and call it. However, the exact code to do so will depend on the compiler, the ABI and the standard C++ library being used. You can't magically reconstruct that code given only the std::function wrapper.
Further, an arbitrary std::function... (or a lambda) may wrap code other than a call to C-style handler. It should be obvious that applying the "magically reconstructed" code to extract C-style handler from such an instance of std::function... can't possibly succeed.
P.S.
To put it differently, what I want is to be able to pass a stateful function to a c callback handler without having to manually instantiate my own struct type to pass along with it. Obviously std::function has already done this for us
So why don't you just use what std::function has already done for you:
void my_new_cpp_handler(std::function<void()>&& func)
{
func(); // calls some_c_handler(p, data) IFF that is what "func" encodes.
}
If you were to carefully review the documentation for this library, you will find that the
void some_c_handler(void(*func)(void*), void* data);
Invokes func, passing it the data argument.
This is a very common design pattern for C libraries that take a callback function. In addition to the callback function, they also take an additional opaque pointer that is not interpreted by the library, but is blindly forwarded to the func. In other words, the C library invokes
func(data);
You can use this from C++ code to pass an ordinary pointer to any class.
This includes std::function, too.
The trick is that in most situations it will be necessary to use new:
auto *pointer=new std::function< function_type >...
The end result is a pointer that can be passed to the C library, together with a pointer to a "trampoline function":
some_c_handler(&cpp_trampoline, reinterpret_cast<void *>(pointer));
And the trampoline recasts the opaque pointer:
void cpp_trampoline(void *pointer)
{
auto real_pointer=reinterpret_cast<std::function< ... >*>(pointer);
// At this point, you have a pointer to the std::function here.
// Do with it as you wish.
The only detail you will need to square away here is to figure out the correct scope for the dynamically-allocated function pointer, in order to avoid memory leaks.
You can make a wrapper function whose purpose is to simply execute the std::function callback.
void some_c_handler(void(*)(void*), void*) {}
void std_function_caller(void* fn) {
(*static_cast<std::function<void()>*>(fn))();
};
auto make_std_function_caller(std::function<void()>& fn) {
return std::make_pair(std_function_caller, static_cast<void*>(&fn));
}
void my_new_cpp_handler(std::function<void()>&& func) {
const auto p = make_std_function_caller(func);
some_c_handler(p.first, p.second);
}
According to this link, the std::function object has no accessible member that can provide raw access to the pointer. You should probably define a struct that contains a pointer to the function pointer and the object, and a constructor wrapper that stores the pointer's address to the struct before the construction of your std::struct, so as to assign the address stored in the pointer it points to to your C handler's parameter.
I have a piece of code which calls an async rdma-write. The rdma API receives a void* context which I would like to use to pass a callback to be called when the operation finishes.
void invoke_async_operation(... some stuff to capture ...) {
...
MyCallBackType* my_callback = // Create callback somehow
rdma_post_write(..., my_callback);
...
}
void on_complete(void* context) {
(*(MyCallbackType*)context)();
}
I thought using a lambda here would be best, because it will easily capture all the context which is required to the later callback invokement. However I saw in What is the lifetime of a C++ lambda expression? that a lambda lifetime is limited to the scope where it was defined.
Note that I can't copy the lambda, because context is a pointer.
What is the correct approach here? Should I insist on using lambdas and prolong their lifetime somehow, or is there a better way? Thanks.
Lifetime of a lambda
The object that represents the lamda expression and allows to invoke it, obeys indeed the usual scoping rules.
However this object can be copied (e.g passing it as argument to a function or a constructor, or assigning it to a global, or whatever else you want to do) so that the lambda can be invoked at any later point, even after the scope it was initially defined in is left.
Because of exactly this potentially long survival of lambdas, you can find quite a few questions, blogs or books that will advise on careful use of the lambda capture, especially if captured by reference, because the lambda itself (and not its anonymous proxy object) can be called even after the referred objects are destroyed.
Your callback issue
You are constraint in your design by the use of an OS callback that can only convey a raw pointer that was passed to it when the callback was set up.
The way to approach this could be to use a std::function object of the standard <functional> library. Here a small function to show you how it works:
function<void()>* preparatory_work() {
auto l = [](){ cout<< "My lambda is fine !" <<endl; } ; // lambda
function<void ()> f = l; // functor
auto p = new function<void()>(l); // a functor on the heap
l(); // inovke the lambda object
f(); // invoke the functor
(*p)(); // invoike functor via a pointer
return p;
}
Function objects are as handy to use as any other object and as easy to declare as function pointers. They are however much more powerful than function pointers, because they can refer basically to any callable object.
As you see, in the example above, I allocated a function objet with new, and returned its pointer. So you could indeed later invoke this function:
int main() {
auto fcp = preparatory_work(); // fcp is a pointer
(*fcp)();
// and even with casting as you would like
void *x = (void*)fcp;
(*(function<void()>*)x)(); // YES !!!
}
Here an online demo
I am trying to interface with a library written in c, that uses this familiar pattern:
void some_c_handler(void(*func)(void*), void* data);
Now, I want to write a C++ wrapper for this function that looks like this:
void my_new_cpp_handler(std::function<void()>&& func)
{
void (*p)() = foo(func);
void* data = bar(func);
some_c_handler(p, data);
}
Both some_c_handler and my_new_cpp_handler are solving the same problem; they're taking in some kind of function along with some state. But the latter is preferred in that it abstracts much of the implementation details from the user, and allows for simply passing in a lambda object.
So my_new_cpp_handler should take the func parameter it is given, convert it to a function pointer and pass its state on to data.
I don't know enough about the standard, or the implementation of std::function to know if this is even a reasonable request. Can foo and bar exist?
To put it differently, what I want is to be able to pass a stateful function to a c callback handler without having to manually instantiate my own struct type to pass along with it. Obviously std::function has already done this for us, so I'd love to be able to separate the function pointer from the state somehow and pass it onto the c-style handler. Is this possible?
Is this possible?
No.
You can wrap a C-style callback into an std::function<>..., and the compiler will emit code to extract the data and the handler and call it. However, the exact code to do so will depend on the compiler, the ABI and the standard C++ library being used. You can't magically reconstruct that code given only the std::function wrapper.
Further, an arbitrary std::function... (or a lambda) may wrap code other than a call to C-style handler. It should be obvious that applying the "magically reconstructed" code to extract C-style handler from such an instance of std::function... can't possibly succeed.
P.S.
To put it differently, what I want is to be able to pass a stateful function to a c callback handler without having to manually instantiate my own struct type to pass along with it. Obviously std::function has already done this for us
So why don't you just use what std::function has already done for you:
void my_new_cpp_handler(std::function<void()>&& func)
{
func(); // calls some_c_handler(p, data) IFF that is what "func" encodes.
}
If you were to carefully review the documentation for this library, you will find that the
void some_c_handler(void(*func)(void*), void* data);
Invokes func, passing it the data argument.
This is a very common design pattern for C libraries that take a callback function. In addition to the callback function, they also take an additional opaque pointer that is not interpreted by the library, but is blindly forwarded to the func. In other words, the C library invokes
func(data);
You can use this from C++ code to pass an ordinary pointer to any class.
This includes std::function, too.
The trick is that in most situations it will be necessary to use new:
auto *pointer=new std::function< function_type >...
The end result is a pointer that can be passed to the C library, together with a pointer to a "trampoline function":
some_c_handler(&cpp_trampoline, reinterpret_cast<void *>(pointer));
And the trampoline recasts the opaque pointer:
void cpp_trampoline(void *pointer)
{
auto real_pointer=reinterpret_cast<std::function< ... >*>(pointer);
// At this point, you have a pointer to the std::function here.
// Do with it as you wish.
The only detail you will need to square away here is to figure out the correct scope for the dynamically-allocated function pointer, in order to avoid memory leaks.
You can make a wrapper function whose purpose is to simply execute the std::function callback.
void some_c_handler(void(*)(void*), void*) {}
void std_function_caller(void* fn) {
(*static_cast<std::function<void()>*>(fn))();
};
auto make_std_function_caller(std::function<void()>& fn) {
return std::make_pair(std_function_caller, static_cast<void*>(&fn));
}
void my_new_cpp_handler(std::function<void()>&& func) {
const auto p = make_std_function_caller(func);
some_c_handler(p.first, p.second);
}
According to this link, the std::function object has no accessible member that can provide raw access to the pointer. You should probably define a struct that contains a pointer to the function pointer and the object, and a constructor wrapper that stores the pointer's address to the struct before the construction of your std::struct, so as to assign the address stored in the pointer it points to to your C handler's parameter.
While using some local lambda objects in a C++11 function I was tempted to declare them as const static auto lambda = ... just to let the compiler know that there is just one std::function object needed (and possibly optimize the call and/or inline it) but I realized that capturing local values by reference in this circumstance leads to weird behavior.
Consider the following code:
void process(const Data& data, const std::function<void(DataElement&>& lambda) {
...
}
void SomeClass::doSomething()
{
int foo = 0;
const static auto lambda = [&foo] () { .... ++foo; .... }
process(data, lambda);
}
This doesn't work with multiple invocations of doSomething() but the mechanics is not clear.
Is foo bound at the first invocation and then kept bound to a stack address which becomes invalid on successive invocations?
Am I forced so drop static in this circumstance?
Where is this behavior specified in the standard? Considering it's a static variable where is it constructed? Lazily on first invocation of doSomething() (so that the first invocation works) or at program start?
A static function-scope variable is initialised "lazily," when control flow first reaches its declaration. This means that the capture by reference does indeed bind to the foo currently on stack, and when the call terminates, that binding becomes dangling.
Don't try to help the compiler too much; making lambda static looks like a micro-optimisation, with very bad side effects. There's next to no overhead involved in actually creating a closure object, and the compiler can easily inline it regardless of whether it's static or not.
Not to mention the fact that you're not saving on creating the std::function object even with your approach. The type of a lambda expression is an unnamed closure object, not std::function. So even if lambda is static, the std::function object is created in each call anyway (unless the whole thing is inlined).
This doesn't work with multiple invocations of doSomething() but the mechanics is not clear.
This is because foo is allocated on the stack. The exact address of foo depends on the call stack that led to the invocation of doSomething. In other words, the address of foo is likely to be different between function invocations, unless the call stack is exactly the same.
I have a program, where I cannot use the standard std::async and threading mechanisms. Instead I have to code the program like so:
void processor( int argument, std::function<void(int)> callback ) {
int blub = 0;
std::shared_ptr<object> objptr = getObject();
// Function is called later.
// All the internal references are bound here!
auto func = [=, &blub]() {
// !This will fail since blub is accessed by reference!
blub *= 2;
// Since objptr is copied by value it works.
// objptr holds the value of getObject().
objptr->addSomething(blub);
// Finally we need to call another callback to return a value
callback(blub);
};
objptr = getAnotherObject();
// Puts func onto a queue and returns immediately.
// func is executed later.
startProcessing(func);
}
I now would like to know whether I am doing it right or what the best way of using lambdas as asynchronous callbacks is.
EDIT: Added expected behavior to the code comments.
See answer/comments for possible solutions for the problem with blub.
The function object will contain a reference to the local variable blub. As in every other situation in the language, this won't make the local variable live after the function ends.
Copies of all the other captured objects will be stored within the function object, since they're captured-by-value. This means there's no issue with them.
If you want it to live after the function ends, you cannot tie its lifetime to the function: you need dynamic storage duration. A std::unique_ptr can be used to to handle the cleanup of such an object, but it gets a bit annoying because you can't "capture-by-move" into a lambda :S
auto blub = make_unique<int>(0); // [1]
std::shared_ptr<object> objptr = getObject();
// use std::bind to store the unique_ptr with the lambda
auto func = std::bind([=](std::unique_ptr<int>& blub) {
*blub *= 2;
objptr->addSomething(*blub);
callback(*blub);
}, std::move(blub)); // move the unique_ptr into the function object
objptr = getAnotherObject();
// func is not copiable because it holds a unique_ptr
startProcessing(std::move(func)); // move it
As an added note, the old deprecated std::auto_ptr would actually work fine here, because if the lambda captures it by value it gets copied and its strange copy semantics are exactly what's needed.
1. See GOTW #102 for make_unique.