Assume I have a templated MemoryPool class a function create(...) (which returns a pointer to a newly allocated object of type T) and a function destroy(T*) (which destroys and returns the memory back to the pool).
I would like to create a std::unique_ptr that "owns" the pointer created by the pool and returns the pointer to the pool, thus requiring a custom deleter.
The problem is, how do I make this work if the pool contains concrete objects and I want to pass around a std::unique_ptr to an abstract interface of this object.
Here is an example that doesn't compile:
#include <iostream>
#include <memory>
#include <functional>
#include <utility>
template <typename T>
class MemoryPool {
public:
template <typename ... ARGS>
T* create(ARGS&&... args) {
std::cout << "MemoryPool::create()" << std::endl;
return new T(std::forward<ARGS>(args)...);
}
void destroy(T* ptr) {
std::cout << "MemoryPool::destroy()" << std::endl;
delete ptr;
}
};
class ITest {
public:
ITest() {
std::cout << "ITest::ITest()" << std::endl;
}
virtual ~ITest() {
std::cout << "ITest::~ITest()" << std::endl;
}
virtual void sayHello() = 0;
};
class Test :public ITest {
public:
Test() {
std::cout << "Test::Test()" << std::endl;
}
~Test() {
std::cout << "Test::~Test()" << std::endl;
}
void sayHello() override {
std::cout << "Test says hello" << std::endl;
}
};
class ITestOwner {
public:
ITestOwner(std::unique_ptr<ITest> ptr) :
_ptr(std::move(ptr))
{
std::cout << "ITestOwner::ITestOwner()" << std::endl;
}
~ITestOwner() {
std::cout << "ITestOwner::~ITestOwner()" << std::endl;
}
void sayHello() { _ptr->sayHello(); }
private:
std::unique_ptr<ITest> _ptr;
};
int main() {
MemoryPool<Test> pool;
std::unique_ptr<Test, std::function<void(Test*)>> ptr(pool.create(), [&pool](Test* ptr){
std::cout << "Custom Deleter" << std::endl;
pool.destroy(ptr);
});
ITestOwner owner(std::move(ptr));
owner.sayHello();
return 0;
}
Keep in mind that, my real MemoryPool class would actually act as a normal memory pool and not use new/delete as I have done.
In this example, ITestOwner should take over ownership of the std::unique_ptr to a ITest abstract object. Then, when ITestOwner is destroyed, the smart pointer will be destroyed, and the Test object should be returned to the memory pool.
Is there a way to accomplish this?
The code doesn't compile because std::unique_ptr<ITest> is used by ITestOwner while you to forward it std::unique_ptr<Test, std::function<void(Test*)>>. It obviously doesn't compile because std::unique_ptr<ITest> calls delete on ITest instead of calling some complex arbitrary function.
You'd need to use unique_ptr<ITest, function<void(ITest*)>> for it to work and in addition add some unsighty conversion code from function<void(Test*)> to function<void(ITest*)>... I'd say this is simply not good. unique_ptr is designed to be simple and efficient - the destructor is supposed to wrap basic functionality but it isn't convenient enough for complicated purposes.
Basically, unique_ptr is not designed for this task. It is supposed to be lightweight and you already use heavy functionality like std::function that ruins the whole purpose. Instead, you can use shared_ptr which type erases the deleter and hides it - so you'd need nothing to worry about. If you want to still restrict user to unique ownership you can surely find other 3rd party open source libraries that implement the smart pointer you want - there are lots of them.
I would like to have a member function unique-ptr with a static deleter function where the function is known at compiletime and no function pointer is required at allocation. I do not know if it is possible but this test confuses me:
#include <memory>
#include <string>
#include <iostream>
struct Apa {
std::string name;
~Apa() {
std::cout << name << " deleted\n";
}
};
static void staticDeleter(Apa *a) {
std::cout << "static deleter\n";
delete a;
}
int main() {
auto deleter = [] (Apa *a) {
std::cout << "deleter\n";
delete a;
};
{
// This is what I want but as a member of a struct
auto ptr1 = std::unique_ptr<Apa, decltype(deleter)>{new Apa{"Apan"}}; // <----------------
std::cout << sizeof(ptr1) << " -> Apan\n";
}
{
auto ptr2 = std::unique_ptr<Apa, decltype(&staticDeleter)>(new Apa{"Beata"}, staticDeleter);
// When trying to omit this argument this line does not compile ------------^
std::cout << sizeof(ptr2) << " -> Beta\n";
}
}
This results in the output
8 -> Apan
deleter
Apan deleted
16 -> Beta
static deleter
Beata deleted
So unless there is some unknown compiler magic, the size of the pointers indicate that it is in fact possible to create a unique ptr with static deleter function.
When trying to do the same thing inside a struct
#include <memory>
#include <iostream>
struct Apa{
};
struct Container {
using deleter = decltype((Apa*) {}); // Edit: This was a typo: se below
std::unique_ptr<Apa, deleter> ptr ;
};
int main() {
std::cout << sizeof(Container::ptr) << "\n";
}
The output 16 shows that the unique_ptr does want a pointer to the deleter function.
Is there any way to make this work so that I define a unique_ptr with a custom deleter function inside a class that is statically defined instead of specified at runtime?
Edit:
After some feedback from #eerorika i realized that the code actually works if the lambda is written correctly. Working code:
#include <memory>
#include <iostream>
struct Apa{
~Apa() {
std::cout << "Apa deleted\n";
}
};
struct Container {
using deleter = decltype([](Apa* apa) {
std::cout << "deleter\n";
delete apa;
});
std::unique_ptr<Apa, deleter> ptr = std::unique_ptr<Apa, deleter>{new Apa} ;
};
int main() {
std::cout << sizeof(Container::ptr) << "\n";
Container container{};
}
Yields output:
8
deleter
Apa deleted
using deleter = decltype((Apa*) {});
This is an unnecessarily convoluted way of writing:
using deleter = Apa*;
The type Apa* doesn't satisfy the requirement that std::unique_ptr imposes on its deleter because Apa* isn't a FunctionObject. The example program is ill-formed (at least if you try to create an instance of it). Hence, it doesn't matter what size the object is.
Is there any way to make this work so that I define a unique_ptr with a custom deleter function inside a class that is statically defined instead of specified at runtime?
Your lambda example works since C++20. You can define similar deleter as a class, which works with earlier versions as well. For example:
struct deleter {
void operator()(Apa* ptr) {
std::cout << "custom deleter\n";
delete ptr;
}
};
struct Container {
std::unique_ptr<Apa, deleter> ptr ;
};
I need to provide a utility function where both memory allocation and object construction take place and the user gets the pointer in return. Similarly, there is a need to provide a utility function for deallocation and destruction also.
Here is what I implemented: (I have not implemented for array type though)
#include <cstdlib> // for malloc
#include <iostream> // for cout
#include <string> // for memset
// C++ class with some initial state x = 10
class SomeClass {
public:
SomeClass() : x(10) {}
int x = 0;
};
template<typename T, typename ... Args>
inline T* MY_MODULE_NEW(Args&&... args) {
// some custom allocator logic here which allocates just raw memory
// for example purpose just use malloc
void *p = malloc(sizeof(T));
memset(p,0,sizeof(T));
T* t = new(p) T(std::forward<Args>(args)...);
return t;
}
template<typename T>
inline void MY_MODULE_DELETE(T* ptr) {
ptr->~T();
// some custom allocator logic here, which deallocates raw memory
// for example purpose just use free
free(ptr);
}
int main() {
SomeClass* sc = MY_MODULE_NEW<SomeClass>();
std::cout << sc->x << std::endl;
SomeClass* sc2 = MY_MODULE_NEW<SomeClass>(*sc);
std::cout << sc2->x << std::endl;
MY_MODULE_DELETE(sc);
MY_MODULE_DELETE(sc2);
}
I have the following concerns:
From the performance point of view, Is this inline function good enough? Can we do better?
Personally I feel MY_MODULE_NEW<SomeClass>(...) syntax is almost similar to the canonical syntax for operator new which is new SomeClass(). Is there any other idiomatic way to achieve the same result?
Thank You!
Recently I tried to reinvent scope guard via std::unique_ptr (NOTE: Deleter has the member typedef pointer — is a specially handled case of std::unique_ptr):
#include <type_traits>
#include <utility>
#include <memory>
#include <iostream>
#include <cstdlib>
#include <cassert>
namespace
{
template< typename lambda >
auto
make_scope_guard(lambda && _lambda)
{
struct lambda_caller
{
using pointer = std::decay_t< lambda >;
void
operator () (lambda & l) const noexcept
{
std::forward< lambda >(l)();
}
};
return std::unique_ptr< std::decay_t< lambda >, lambda_caller >(std::forward< lambda >(_lambda));
}
}
int
main()
{
std::cout << 1 << std::endl;
{
std::cout << 2 << std::endl;
[[gnu::unused]] auto && guard_ = make_scope_guard([&] { std::cout << __PRETTY_FUNCTION__ << std::endl; });
std::cout << 3 << std::endl;
}
std::cout << 5 << std::endl;
return EXIT_SUCCESS;
}
Such an approach works fine for simple pointer to free function void f() { std::cout << 4 << std::endl; } passed to make_scope_guard, but not for any lambda passed to make_scope_guard.
This is due to an abundance of ... = pointer() into the std::unique_ptr definition (function default parameter, defaulting data memebers etc), but I can't find the DefaultConstructible requirement for pointer into this article.
Is it mandatory, that the pointer should match the std::is_default_constructible requirement?
It tested against libc++ and against libstdc++ using not too old clang++ -std=gnu++1z.
Seems, there should be language extension for lambdas: if auto l = [/* possible capture list */] (Args...) { /* code */; }; then using L = decltype(l); is equivalent to struct L { constexpr void operator () (Args...) const noexcept { ; } }; for some Args..., isn't it?
ADDITIONAL:
Providing the instance D{} of following DefaultConstructible class to make_scope_guard(D{}) requires commented out code to be uncommented in the context if (p) { ..., where p is of type D:
struct D { void operator () () const noexcept { std::cout << __PRETTY_FUNCTION__ << std::endl; } /* constexpr operator bool () const { return true; } */ };
A unique_ptr is still a pointer. You cannot shoehorn a lambda into it. From [unique.ptr]:
A unique pointer is an object that owns another object and manages that other object through a pointer.
More precisely, a unique pointer is an object u that stores a pointer to a second object p and will dispose of
p when u is itself destroyed
[...]
Additionally, u can, upon request, transfer ownership to another unique pointer u2. Upon completion of
such a transfer, the following post-conditions hold: [...] u.p is equal to nullptr
A lambda is not a pointer. A lambda cannot equal nullptr.
That said, you're already making your own local struct, why not just use that to do the RAII scope guarding itself instead of deferring to unique_ptr? That seems like a hack at best, and takes more code to boot. You could instead just do:
template< typename lambda >
auto
make_scope_guard(lambda && _lambda)
{
struct lambda_caller
{
lambda _lambda;
~lambda_caller()
{
_lambda();
}
};
return lambda_caller{std::forward<lambda>(_lambda)};
}
If you need to support release, you can wrap _lambda inside of boost::optional so that lambda_caller becomes:
struct lambda_caller
{
boost::optional<lambda> _lambda;
~lambda_caller()
{
if (_lambda) {
(*_lambda)();
_lambda = boost::none;
}
}
void release() {
_lambda = boost::none;
}
};
I am currently working with Boost.Python and would like some help to solve a tricky problem.
Context
When a C++ method/function is exposed to Python, it needs to release the GIL (Global Interpreter Lock) to let other threads use the interpreter. This way, when the python code calls a C++ function, the interpreter can be used by other threads.
For now, each C++ function looks like this:
// module.cpp
int myfunction(std::string question)
{
ReleaseGIL unlockGIL;
return 42;
}
To pass it to boost python, I do:
// python_exposure.cpp
BOOST_PYTHON_MODULE(PythonModule)
{
def("myfunction", &myfunction);
}
Problem
This scheme works fine, however it implies that module.cpp depends on Boost.Python for no good reason. Ideally, only python_exposure.cpp should depend on Boost.Python.
Solution?
My idea was to play with Boost.Function to wrap the function calls like this:
// python_exposure.cpp
BOOST_PYTHON_MODULE(PythonModule)
{
def("myfunction", wrap(&myfunction));
}
Here wrap would be in charge of unlocking the GIL during the call to myfunction. The problem with this method is that wrap needs to have the same signature as myfunction which would pretty much mean re-implementing Boost.Function...
I would be very thankful if someone had any suggestion to this problem.
Exposing functors as methods is not officially supported. The supported approach would be to expose a non-member function that delegates to the member-function. However, this can result in a large amount of boilerplate code.
As best as I can tell, Boost.Python's implementation does not explicitly preclude functors, as it allows for instances of python::object to be exposed as a method. However, Boost.Python does place some requirements on the type of object being exposed as a method:
The functor is CopyConstructible.
The functor is callable. I.e. instance o can be called o(a1, a2, a3).
The call signature must be available as meta-data during runtime. Boost.Python calls the boost::python::detail::get_signature() function to obtain this meta-data. The meta-data is used internally to setup proper invocation, as well as for dispatching from Python to C++.
The latter requirement is where it gets complex. For some reason that is not immediately clear to me, Boost.Python invokes get_signature() through a qualified-id, preventing argument dependent lookup. Therefore, all candidates for get_signature() must be declared before the calling template's definition context. For example, the only overloads for get_signature() that are considered are those declared before the definition of templates that invoke it, such as class_, def(), and make_function(). To account for this behavior, when enabling a functor in Boost.Python, one must provide a get_signature() overload prior to including Boost.Python or explicitly provide a meta-sequence representing the signature to make_function().
Lets work through some examples of enabling functor support, as well as providing functors that support guards. I have opted to not use C++11 features. As such, there will be some boilerplate code that could be reduced with variadic templates. Additionally, all of the examples will use the same model that provides two non-member functions and a spam class that has two member-functions:
/// #brief Mockup class with member functions.
class spam
{
public:
void action()
{
std::cout << "spam::action()" << std::endl;
}
int times_two(int x)
{
std::cout << "spam::times_two()" << std::endl;
return 2 * x;
}
};
// Mockup non-member functions.
void action()
{
std::cout << "action()" << std::endl;
}
int times_two(int x)
{
std::cout << "times_two()" << std::endl;
return 2 * x;
}
Enabling boost::function
When using the preferred syntax for Boost.Function, decomposing the signature into meta-data that meets Boost.Python requirements can be done with Boost.FunctionTypes. Here is a complete example enabling boost::function functors to be exposed as a Boost.Python method:
#include <iostream>
#include <boost/function.hpp>
#include <boost/function_types/components.hpp>
namespace boost {
namespace python {
namespace detail {
// get_signature overloads must be declared before including
// boost/python.hpp. The declaration must be visible at the
// point of definition of various Boost.Python templates during
// the first phase of two phase lookup. Boost.Python invokes the
// get_signature function via qualified-id, thus ADL is disabled.
/// #brief Get the signature of a boost::function.
template <typename Signature>
inline typename boost::function_types::components<Signature>::type
get_signature(boost::function<Signature>&, void* = 0)
{
return typename boost::function_types::components<Signature>::type();
}
} // namespace detail
} // namespace python
} // namespace boost
#include <boost/python.hpp>
/// #brief Mockup class with member functions.
class spam
{
public:
void action()
{
std::cout << "spam::action()" << std::endl;
}
int times_two(int x)
{
std::cout << "spam::times_two()" << std::endl;
return 2 * x;
}
};
// Mockup non-member functions.
void action()
{
std::cout << "action()" << std::endl;
}
int times_two(int x)
{
std::cout << "times_two()" << std::endl;
return 2 * x;
}
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
// Expose class and member-function.
python::class_<spam>("Spam")
.def("action", &spam::action)
.def("times_two", boost::function<int(spam&, int)>(
&spam::times_two))
;
// Expose non-member function.
python::def("action", &action);
python::def("times_two", boost::function<int()>(
boost::bind(×_two, 21)));
}
And its usage:
>>> import example
>>> spam = example.Spam()
>>> spam.action()
spam::action()
>>> spam.times_two(5)
spam::times_two()
10
>>> example.action()
action()
>>> example.times_two()
times_two()
42
When providing a functor that will invoke a member-function, the provided signature needs to be the non-member function equivalent. In this case, int(spam::*)(int) becomes int(spam&, int).
// ...
.def("times_two", boost::function<int(spam&, int)>(
&spam::times_two))
;
Also, arguments can be bound to the functors with boost::bind. For example, calling example.times_two() does not have to provide an argument, as 21 is already bound to the functor.
python::def("times_two", boost::function<int()>(
boost::bind(×_two, 21)));
Custom functor with guards
Expanding upon the above example, one can enable custom functor types to be used with Boost.Python. Lets create a functor, called guarded_function, that will use RAII, only invoking the wrapped function during the RAII object's lifetime.
/// #brief Functor that will invoke a function while holding a guard.
/// Upon returning from the function, the guard is released.
template <typename Signature,
typename Guard>
class guarded_function
{
public:
typedef typename boost::function_types::result_type<Signature>::type
result_type;
template <typename Fn>
guarded_function(Fn fn)
: fn_(fn)
{}
result_type operator()()
{
Guard g;
return fn_();
}
// ... overloads for operator()
private:
boost::function<Signature> fn_;
};
The guarded_function provides similar semantics to the Python with statement. Thus, to keep with the Boost.Python API name choices, a with() C++ function will provide a way to create functors.
/// #brief Create a callable object with guards.
template <typename Guard,
typename Fn>
boost::python::object
with(Fn fn)
{
return boost::python::make_function(
guarded_function<Guard, Fn>(fn), ...);
}
This allows for functions to be exposed which will run with a guard in a non-intrusive manner:
class no_gil; // Guard
// ...
.def("times_two", with<no_gil>(&spam::times_two))
;
Additionally, the with() function provides the ability to deduce the function signatures, allowing the meta-data signature to be explicitly provided to Boost.Python rather than having to overload boost::python::detail::get_signature().
Here is the complete example, using two RAII types:
no_gil: Releases GIL in constructor, and reacquires GIL in destructor.
echo_guard: Prints in constructor and destructor.
#include <iostream>
#include <boost/function.hpp>
#include <boost/function_types/components.hpp>
#include <boost/function_types/function_type.hpp>
#include <boost/function_types/result_type.hpp>
#include <boost/python.hpp>
#include <boost/tuple/tuple.hpp>
namespace detail {
/// #brief Functor that will invoke a function while holding a guard.
/// Upon returning from the function, the guard is released.
template <typename Signature,
typename Guard>
class guarded_function
{
public:
typedef typename boost::function_types::result_type<Signature>::type
result_type;
template <typename Fn>
guarded_function(Fn fn)
: fn_(fn)
{}
result_type operator()()
{
Guard g;
return fn_();
}
template <typename A1>
result_type operator()(A1 a1)
{
Guard g;
return fn_(a1);
}
template <typename A1, typename A2>
result_type operator()(A1 a1, A2 a2)
{
Guard g;
return fn_(a1, a2);
}
private:
boost::function<Signature> fn_;
};
/// #brief Provides signature type.
template <typename Signature>
struct mpl_signature
{
typedef typename boost::function_types::components<Signature>::type type;
};
// Support boost::function.
template <typename Signature>
struct mpl_signature<boost::function<Signature> >:
public mpl_signature<Signature>
{};
/// #brief Create a callable object with guards.
template <typename Guard,
typename Fn,
typename Policy>
boost::python::object with_aux(Fn fn, const Policy& policy)
{
// Obtain the components of the Fn. This will decompose non-member
// and member functions into an mpl sequence.
// R (*)(A1) => R, A1
// R (C::*)(A1) => R, C*, A1
typedef typename mpl_signature<Fn>::type mpl_signature_type;
// Synthesize the components into a function type. This process
// causes member functions to require the instance argument.
// This is necessary because member functions will be explicitly
// provided the 'self' argument.
// R, A1 => R (*)(A1)
// R, C*, A1 => R (*)(C*, A1)
typedef typename boost::function_types::function_type<
mpl_signature_type>::type signature_type;
// Create a callable boost::python::object that delegates to the
// guarded_function.
return boost::python::make_function(
guarded_function<signature_type, Guard>(fn),
policy, mpl_signature_type());
}
} // namespace detail
/// #brief Create a callable object with guards.
template <typename Guard,
typename Fn,
typename Policy>
boost::python::object with(const Fn& fn, const Policy& policy)
{
return detail::with_aux<Guard>(fn, policy);
}
/// #brief Create a callable object with guards.
template <typename Guard,
typename Fn>
boost::python::object with(const Fn& fn)
{
return with<Guard>(fn, boost::python::default_call_policies());
}
/// #brief Mockup class with member functions.
class spam
{
public:
void action()
{
std::cout << "spam::action()" << std::endl;
}
int times_two(int x)
{
std::cout << "spam::times_two()" << std::endl;
return 2 * x;
}
};
// Mockup non-member functions.
void action()
{
std::cout << "action()" << std::endl;
}
int times_two(int x)
{
std::cout << "times_two()" << std::endl;
return 2 * x;
}
/// #brief Guard that will unlock the GIL upon construction, and
/// reacquire it upon destruction.
struct no_gil
{
public:
no_gil() { state_ = PyEval_SaveThread();
std::cout << "no_gil()" << std::endl; }
~no_gil() { std::cout << "~no_gil()" << std::endl;
PyEval_RestoreThread(state_); }
private:
PyThreadState* state_;
};
/// #brief Guard that prints to std::cout.
struct echo_guard
{
echo_guard() { std::cout << "echo_guard()" << std::endl; }
~echo_guard() { std::cout << "~echo_guard()" << std::endl; }
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
// Expose class and member-function.
python::class_<spam>("Spam")
.def("action", &spam::action)
.def("times_two", with<no_gil>(&spam::times_two))
;
// Expose non-member function.
python::def("action", &action);
python::def("times_two", with<boost::tuple<no_gil, echo_guard> >(
×_two));
}
And its usage:
>>> import example
>>> spam = example.Spam()
>>> spam.action()
spam::action()
>>> spam.times_two(5)
no_gil()
spam::times_two()
~no_gil()
10
>>> example.action()
action()
>>> example.times_two(21)
no_gil()
echo_guard()
times_two()
~echo_guard()
~no_gil()
42
Notice how multiple guards can be provided by using a container type, such as boost::tuple:
python::def("times_two", with<boost::tuple<no_gil, echo_guard> >(
×_two));
When invoked in Python, example.times_two(21) produces the following output:
no_gil()
echo_guard()
times_two()
~echo_guard()
~no_gil()
42
If someone is interested, I had a small issue with Tanner Sansbury's code when using his final working example. For some reason, I still had the problem he mentioned about having the wrong signature in the final generated boost::function:
// example for spam::times_two:
// correct signature (manual)
int (spam::*, int)
// wrong signature (generated in the `guarded_function` wrapper)
int (spam&, int)
even when overloading boost::python::detail::get_signature(). The responsible for this was boost::function_types::components; it has a default template parameter ClassTranform = add_reference<_> which creates this class reference. To fix this, I simply changed the mpl_signature struct as follow:
// other includes
# include <boost/type_traits/add_pointer.hpp>
# include <boost/mpl/placeholders.hpp>
template <typename Signature> struct mpl_signature
{
typedef typename boost::function_types::components<Signature,
boost::add_pointer<boost::mpl::placeholders::_> >::type type;
};
template <typename Signature> struct mpl_signature<boost::function<Signature> >
{
typedef typename boost::function_types::components<Signature>::type type;
};
And now everything works like a charm.
If someone can confirm this is indeed the right fix, I'd be interested :)