Memory requirements for C++ lambdas - c++

Question

C++ lambdas produce an std::function. In order be able to execute the lambda later, all this state needs to be stored somewhere. So where are these captured values stored and what allocator requirements must be present for this to work?
Also is there any way to vary this behaviour if desired?

Answer 1

A lambda expression does not create a std::function object. Instead, it creates an object (a closure object) of an unnamed class type called the closure type. That closure type will have the following components:
operator()
One member variable for each capture by copy
Potentially member variables for captures by reference
If it has no captures, a conversion operator to a pointer to function
Auto-generate copy constructor
Potentially an auto-generated move constructor
It is guaranteed to have:
No default constructor
No copy/move assignment operator
Nothing else is guaranteed by the standard. The closure type's member variables, like any other member variables, have automatic storage duration; that is, they're contained inside the closure object itself. There are no special allocator requirements, and there is no way to change how these members are managed.

Answer 2

Lambdas are an unnamed type that can be stored in a std::function. Important distinction there. You can think of the declaration of a lambda as very similar to the declaration of a functor class. Example:
std::vector<int> vec(......);
auto& lambda = [vec](int x) -> void {};
We created a copy of vec and receive an int at call time on the stack. Your lambda in this case is roughly equivalent to this class
class mylambda
{
public:
mylambda(const std::vector& vecin) : vec(vecin) {}
void operator() (int x) const {}
private:
std::vector<int> vec;
};
Note that your by-value captures are copied in. Reference captures are stored essentially as pointers. In all likelihood your compiler will defer construction of the lambda for as long as possible. Eventually though, the captures will be stored in the object.
Back to std::function - if you choose to copy the lambda into a function object, because you maybe want to pass it outside the owning function, then your lambda object will be copied (or moved, depending on your syntax) into the function object. In the example above, your function object would indeed store a vector, though keep in mind that the vector stores its payload on the heap (possibly using a custom allocator).

Answer 3

C++ lambdas produce an std::function.
No they don't. They produce an object of class type, with an overloaded operator() so that they can be called like a function.
std::function is a wrapper for any callable type, including lambdas.
In order be able to execute the lambda later, all this state needs to be stored somewhere.
Indeed, captured values need to be stored in the lambda.
So where are these captured values stored and what allocator requirements must be present for this to work?
Captured values are stored as members of the lambda class. References to local variables might be treated similarly; or might be optimised to capture a single pointer to the stack frame containing them.
Also is there any way to vary this behaviour if desired?
No.

Answer 4

Beside the confusion between lambdas and std::function, at least part of your question can be rephrased as:
where does std::function stores its state?".
Either in a small buffer inside std::function itself, or it will allocate a chunk of memory using the default allocator to store the state.
Although it is largely unspecified whether a generic object of type T will fit the first or the second category, std::reference_wrapper is guaranteed to make use of small buffer optimization; that is, the following line:
std::function<void()> f = std::ref( function_object );
does not contain any heap allocation.
Also is there any way to vary this behaviour if desired?
Take a look at the definition of the std::function constructors: most of them take an additional Allocator parameter, that allows you to specify a custom allocator. I assume that the allocator, if stateful, will get copied and type-erased inside the memory provided by the allocator itself.