I have a program, where I cannot use the standard std::async and threading mechanisms. Instead I have to code the program like so:
void processor( int argument, std::function<void(int)> callback ) {
int blub = 0;
std::shared_ptr<object> objptr = getObject();
// Function is called later.
// All the internal references are bound here!
auto func = [=, &blub]() {
// !This will fail since blub is accessed by reference!
blub *= 2;
// Since objptr is copied by value it works.
// objptr holds the value of getObject().
objptr->addSomething(blub);
// Finally we need to call another callback to return a value
callback(blub);
};
objptr = getAnotherObject();
// Puts func onto a queue and returns immediately.
// func is executed later.
startProcessing(func);
}
I now would like to know whether I am doing it right or what the best way of using lambdas as asynchronous callbacks is.
EDIT: Added expected behavior to the code comments.
See answer/comments for possible solutions for the problem with blub.
The function object will contain a reference to the local variable blub. As in every other situation in the language, this won't make the local variable live after the function ends.
Copies of all the other captured objects will be stored within the function object, since they're captured-by-value. This means there's no issue with them.
If you want it to live after the function ends, you cannot tie its lifetime to the function: you need dynamic storage duration. A std::unique_ptr can be used to to handle the cleanup of such an object, but it gets a bit annoying because you can't "capture-by-move" into a lambda :S
auto blub = make_unique<int>(0); // [1]
std::shared_ptr<object> objptr = getObject();
// use std::bind to store the unique_ptr with the lambda
auto func = std::bind([=](std::unique_ptr<int>& blub) {
*blub *= 2;
objptr->addSomething(*blub);
callback(*blub);
}, std::move(blub)); // move the unique_ptr into the function object
objptr = getAnotherObject();
// func is not copiable because it holds a unique_ptr
startProcessing(std::move(func)); // move it
As an added note, the old deprecated std::auto_ptr would actually work fine here, because if the lambda captures it by value it gets copied and its strange copy semantics are exactly what's needed.
1. See GOTW #102 for make_unique.
Related
I have always assumed lambda were just function pointers, but I've never thought to use capture statements seriously...
If I create a lambda that captures by copy, and then move that lambda to a completely different thread and make no attempt to save the original objects used in the lambda, will it retain those copies for me?
std::thread createThread() {
std::string str("Success");
auto func = [=](){
printf("%s", str.c_str());
};
str = "Failure";
return std::thread(func);
}
int main() {
std::thread thread = createThread();
thread.join();
// assuming the thread doesn't execute anything until here...
// would it print "Success", "Failure", or deference a dangling pointer?
return 0;
}
It is guaranteed to print Success. Capture-by-copy does exactly what it says. It make a copy of the object right there and stores this copy as part of the closure object. The member of the closure object created from the capture lives as long as the closure object itself.
A lambda is not a function pointer. Lambdas are general function objects that can have internal state, which a function pointer can't have. In fact, only capture-less lambdas can be converted to function pointers and so may behave like one sometimes.
The lambda expression produces a closure type that basically looks something like this:
struct /*unnamed1*/ {
/*unnamed1*/(const /*unnamed1*/&) = default;
/*unnamed1*/(/*unnamed1*/&&) = default;
/*unnamed1*/& operator=(const /*unnamed1*/&) = delete;
void operator()() const {
printf("%s", /*unnamed2*/.c_str());
};
std::string /*unnamed2*/;
};
and the lambda expression produces an object of this type, with /*unnamed2*/ direct-initialized to the current value of str. (Direct-initialized meaning as if by std::string /*unnamed2*/(str);)
You have 3 situations
You can be design guarantee that variables live longer then the thread, because you synchronize with the end of the thread before variables go out of scope.
You know your thread may outlive the scope/life cycle of your thread but you don't need access to the variables anymore from any other thread.
You can't say which thread lives longest, you have multiple thread accessing your data and you want to extend the live time of your variables
In case 1. Capture by reference
In case 2. Capture by value (or you even use move) variables
In case 3. Make data shared, std::shared_ptr and capture that by value
Case 3 will extend the lifetime of the data to the lifetime of the longest living thread.
Note I prefer using std::async over std::thread, since that returns a RAII object (a future). The destructor of that will synchronize with the thread. So you can use that as members in objects with a thread and make sure the object destruction waits for the thread to finish.
I am working with a large object that is stored in a shared memory region, and is returned by a getter. I was running into issues where returning the object by-value was blowing the stack due to the size of the object. To avoid this, I decided to return a unique_ptr to a copy of the object on the heap, which I want the calling function to be able to use as a local variable. See my example code below:
Note: The codebase I am working on is C++11, not C++14, so I cannot use std::make_unique
#include <memory>
struct AbsoluteUnit {
char bigChungus[10000000];
int bigChungusSize;
};
AbsoluteUnit g_absUnit;
std::unique_ptr<AbsoluteUnit> getAbsoluteUnit() {
return std::move(std::unique_ptr<AbsoluteUnit>(new AbsoluteUnit(g_absUnit)));
}
void Example1()
{
std::unique_ptr<AbsoluteUnit> absUnitPtr = getAbsoluteUnit();
AbsoluteUnit& absUnit = *absUnitPtr.get();
///
/// Some other code
///
}
void Example2()
{
AbsoluteUnit& absUnit = *getAbsoluteUnit().get();
///
/// Some other code
///
}
My question is: In Example2, when does the unique_ptr go out of scope?
In Example1 I am storing the unique_ptr in a local variable, so I would expect it to go out of scope when the function exits. But in Example2, I am not creating a local variable for it, so what happens to the unique_ptr after that line executes? Is it still scoped to the function, or does it immediately go out of scope?
The get() member function returns the pointer to the AbsoluteUnit object managed by the std::unique_ptr<AbsoluteUnit> that getAbsoluteUnit() returns.
That managed AbsoluteUnit object is then bound to the reference absUnit:
AbsoluteUnit& absUnit = *getAbsoluteUnit().get();
However, the std::unique_ptr ceases to exist just after this statement above, and therefore so does the managed AbsoluteUnit object. So, absUnit becomes a dangling reference.
If you want to avoid working with the pointer syntax, you could write instead:
const std::unique_ptr<AbsoluteUnit> ptr = getAbsoluteUnit();
AbsoluteUnit& obj = *ptr;
// ... use obj instead of ptr
or even more concise with auto:
const auto ptr = getAbsoluteUnit();
auto& obj = *ptr;
// ... use obj instead of ptr
I'm trying to get my head around this code that I'm looking at:
class ShipFactory
{
public:
Ship make_ship(....) { ... }
static std::unique_ptr<ShipFactory>&& get_factory()
{
if (!m_factory) {
m_factory.reset(new ShipFactory);
}
return std::move(m_factory);
}
...
public:
static std::unique_ptr<ShipFactory> m_factory;
};
std::unique_ptr<ShipFactory> ShipFactory::m_factory;
...
// used like this:
ship = ShipFactory::get_factory()->make_ship(...);
My question is about the get_factory method. I don't really understand why it's returning an rvalue reference to a std::unique_ptr or what that will do. I'm also not entirely convinced it's valid.
Unlike auto_ptr, you can't copy a unique_ptr since that constructor is deleted and that would make no sense for the unique_ptr since it owns the memory it points to, but you can move it. This is what's happening here, resets a unique_ptr, and moves it. It has the same effect as:
auto ship = make_unique<ShipFactory>();
Remember that std::move doesn’t move anything; it merely allows a move to happen (and causes rvalue references that (presumably) move to be preferred in overload resolution).
Returning an rvalue reference (to something that is not destroyed when leaving the function!) is an offer of the object’s resources to the client:
auto s=ShipFactory::get_factory()->make_ship(…); // use the current factory
auto f=ShipFactory::get_factory(); // take the whole current factory
Both lines create a factory if there is no current factory; the second causes there to be no current factory thereafter (but the unique_ptr f can still be used).
I am not gonna pretend I am very experienced with lambdas, but with this seemingly simple issue, I think I need deeper insight on how it actually works.
I am instantiating an object on an outer scope, and using 2 lambdas, I attempt to modify that pointer.
// Object to be manipulated
Object* obj= nullptr;
// To be invoked externally in order to construct and initialize obj
std::function<void(std::function<void(const String&)>)> SetObject
= [&obj](std::function<void(const String&)> func)
{
obj= new Object();
// ... Initialize Values
};
// To be invoked externally to reset and delete obj
std::function<void()> ResetObject
= [&obj]()
{
if(obj)
{
delete obj;
obj= nullptr;
}
};
A main code executes and first calls the RESET method before creating a new instance of Object. Assume the "main" code has no way of accessing, reading, or modifying obj - hence the reliance on a blind reset.
Expected Result: The first time the reset expression is invoked, the check on obj should return false because obj was set to nullptr in the outer scope -- nothing happens.
Actual Result: By the time reset is executed, obj no longer points to NULL, the check returns true and delete is called, causing an access violation.
I would like to know if the whatever I am attempting to do is an invalid operation to begin with, or whether the way I am capturing my variables is incorrect. I tried capturing the pointer by value, setting the function to mutable, and a combination of reference and value capture out of desperation.
EDIT: At this point, I believe the scope of the object is the main culprit as pointed out by others. I will attempt a workaround with
that in mind and report back.
While we cannot tell from your example alone, I suspect obj is perhaps a local variable which goes out of scope after being captured. One way to solve this would be to add another level of indirection:
Object** objptr = new Object*();
std::function<void(std::function<void(const String&)>)> SetObject
= [objptr](std::function<void(const String&)> func)
{
*objptr = new Object();
};
std::function<void()> ResetObject
= [objptr]() // Note we are now capturing objptr by value
{
if(*objptr)
{
delete *objptr;
*objptr = nullptr;
}
// Is objptr reused by another call to SetObject?
// If not, delete it here as well.
// delete objptr;
};
I have a piece of code which calls an async rdma-write. The rdma API receives a void* context which I would like to use to pass a callback to be called when the operation finishes.
void invoke_async_operation(... some stuff to capture ...) {
...
MyCallBackType* my_callback = // Create callback somehow
rdma_post_write(..., my_callback);
...
}
void on_complete(void* context) {
(*(MyCallbackType*)context)();
}
I thought using a lambda here would be best, because it will easily capture all the context which is required to the later callback invokement. However I saw in What is the lifetime of a C++ lambda expression? that a lambda lifetime is limited to the scope where it was defined.
Note that I can't copy the lambda, because context is a pointer.
What is the correct approach here? Should I insist on using lambdas and prolong their lifetime somehow, or is there a better way? Thanks.
Lifetime of a lambda
The object that represents the lamda expression and allows to invoke it, obeys indeed the usual scoping rules.
However this object can be copied (e.g passing it as argument to a function or a constructor, or assigning it to a global, or whatever else you want to do) so that the lambda can be invoked at any later point, even after the scope it was initially defined in is left.
Because of exactly this potentially long survival of lambdas, you can find quite a few questions, blogs or books that will advise on careful use of the lambda capture, especially if captured by reference, because the lambda itself (and not its anonymous proxy object) can be called even after the referred objects are destroyed.
Your callback issue
You are constraint in your design by the use of an OS callback that can only convey a raw pointer that was passed to it when the callback was set up.
The way to approach this could be to use a std::function object of the standard <functional> library. Here a small function to show you how it works:
function<void()>* preparatory_work() {
auto l = [](){ cout<< "My lambda is fine !" <<endl; } ; // lambda
function<void ()> f = l; // functor
auto p = new function<void()>(l); // a functor on the heap
l(); // inovke the lambda object
f(); // invoke the functor
(*p)(); // invoike functor via a pointer
return p;
}
Function objects are as handy to use as any other object and as easy to declare as function pointers. They are however much more powerful than function pointers, because they can refer basically to any callable object.
As you see, in the example above, I allocated a function objet with new, and returned its pointer. So you could indeed later invoke this function:
int main() {
auto fcp = preparatory_work(); // fcp is a pointer
(*fcp)();
// and even with casting as you would like
void *x = (void*)fcp;
(*(function<void()>*)x)(); // YES !!!
}
Here an online demo