I'm trying to add a python callback to a C++ library as illustrated:
template<typename T> void doCallback(shared_ptr<T> data) {
PyObject* pyfunc; //I have this already
PyObject* args = Py_BuildValue("(O)", data);
PyEval_CallObject(pyfunc,args);
}
This fails because data hasn't gone through swig, and isn't a PyObject.
I tried using:
swigData = SWIG_NewPointerObj((void*)data, NULL, 0);
But because its a template, I don't really know what to use for the second parameter. Even if I do hard code the 'correct' SWIGTYPE, it usually segfaults on PyEval_CallObject.
So my questions are:
Whats the best way to invoke swig
type wrapping?
Am I even going in the right
direction here? Directors looked
promising for implementing a
callback, but I couldn't find an
example of directors with python.
Update: The proper wrapping is getting generated. I have other functions that return shared_ptrs and can call those correctly.
My first answer misunderstood the question completely, so let's try this again.
Your central problem is the free type parameter T in the definition of doCallback. As you point out in your question, there's no way to make a SWIG object out of a shared_ptr<T> without a concrete value for T: shared_ptr<T> isn't really a type.
Thus I think that you have to specialize: for each concrete instantiation of doCallback that the host system uses, provide a template specialization for the target type. With that done, you can generate a Python-friendly wrapping of data, and pass it to your python function. The simplest technique for that is probably:
swigData = SWIG_NewPointerObj((void*)(data.get()), SWIGType_Whatever, 0);
...though this can only work if your Python function doesn't save its argument anywhere, as the shared_ptr itself is not copied.
If you do need to retain a reference to data, you'll need to use whatever mechanism SWIG usually uses to wrap shared_ptr. If there's no special-case smart-pointer magic going on, it's probably something like:
pythonData = new shared_ptr<Whatever>(data);
swigData = SWIG_NewPointerObj(pythonData, SWIGType_shared_ptr_to_Whatever, 1);
Regardless, you then you have a Python-friendly SWIG object that's amenable to Py_BuildValue().
Hope this helps.
shared_ptr<T> for unknown T isn't a type, so SWIG can't hope to wrap it. What you need to do is provide a SWIG wrapping for each instance of shared_ptr that you intend to use. So if for example you want to be able to doCallback() with both shared_ptr<Foo> and shared_ptr<Bar>, you will need:
A wrapper for Foo
A wrapper for Bar
Wrappers for shared_ptr<Foo> and shared_ptr<Bar>.
You make those like so:
namespace boost {
template<class T> class shared_ptr
{
public:
T * operator-> () const;
};
}
%template(FooSharedPtr) boost::shared_ptr<Foo>;
%template(BarSharedPtr) boost::shared_ptr<Bar>;
Related
I am relatively new to C++ and I am working on a personal project for practicing where I need to create a class that can take function pointers with different signatures.
The idea is that each instance of the class will store a pointer to a specific function and can call that function whenever I want.
To give a better idea of what I want, let me explain with a little bit more detail what I am trying to do. The project I am working on is a very basic console game and the object I am trying to create is an object that would store details on each location the player can access in the game.
(DISCLAIMER: I know that most of what I describe later is probably an overkill for a basic console game. I can easily make the whole game in a couple of files using just simple functions and I know how to do that. But the idea here is that I wanted to practice more advanced C++ techniques without having to figure out a complex project. So, since I know how to make a basic console game, I thought it would be a good idea to try and figure out how to achieve the same result but with more advanced techniques)
One of the details that I think should be stored is what happens in each location, basically the text that is output to the screen describing what happens and prompting the user to take action.
Since this would be different for each location, I can't just declare and implement a function in the class.
One way of solving this issue is to create a base class with a virtual function and then implement this function in a series of derived classes, each defining a new location.
The problem I have with this approach is that it makes each location a class that can be inherited further and instanced, which I don't need as I will only have 1 instance of each location.
I can of course just create 1 instance of the class, but I wanted to see if there is a way to avoid having to create separate classes for each location.
This why I started thinking of function pointers.
Now, I know I can declare a function pointer and initialise it in a class like that:
class Test
{
public:
Test(void (*p)())
: print{p}
{}
private:
void (*print)();
};
That works fine as long as the function returns void and accepts no arguments.
So, I thought maybe I can do that with a template:
template <typename Function>
class Test
{
public:
Test(Function *p)
: print{p}
{}
Function *print;
};
This actually works well. I can now have a class that accepts different functions with different return types.
I can create instances of the class in the following way:
void print();
Test<void ()> a {print};
However, I have one problem with this approach. Because it is a class template, I can't have a pointer that I want to use to point to instances of Test class regardless of the function that is passed to them.
For instance, if I declare the following pointer:
Test<void ()> *b = &a;
There is no way to re-assign that pointer to another instance of Test class unless the function pointer passed to it also returns void and accepts no arguments. Otherwise, I have to create a new pointer.
Is there a way to avoid that? Is there a better way of achieving what I am looking for than using function pointers?
Thank you very much and sorry for the long message.
There is no way to re-assign that pointer to another instance of Test class unless the function pointer passed to it also returns void and accepts no arguments.
And how would you expect that to work? Each Test has a different signature for its print method. In the example below, if you assigned action2 to action1, then how would it know what to pass to the print function?
void fun1() {}
void fun2(int) {}
void test() {
Test<void()> action1= &fun1;
action1.print();
Test<void(int)> action2= &fun2;
action2.print(42);
}
What you want is really simple, thanks to improvements in C++11: std::function<void()>.
Whenever you need to pass some arguments to the function, you'd use a lambda, potentially with captures:
#include <functional>
void fun1() {}
void fun2(int) {}
using Action = std::function<void()>;
int main() {
class MyObject { public: void doSomething() {} } obj;
Action action1;
action1 = fun1;
action1 = []{ fun2(42); };
action1 = [&obj]{ obj.doSomething(); }; // obj captured by reference
}
So that's pretty much what you should do.
Now of course you may ask "hey, but what if I wanted to implement something like std::function myself"? It's not a trivial task, if you want to get full functionality of std::function, including small object optimization (i.e. std::function usually doesn't allocate unless the functor it wraps is "big").
I wanted to see what can I do if I had a reason to have the each game location using a function that have different return types.
You chose to use a common abstraction, i.e. some class (like Action) that you can always call the same way, and get to do various things. So, what would you like the following code to do?
Action someAction;
int result = someAction();
std::string result = someAction();
Now suppose that e.g. someAction has captured a void() function. What should the code that assigns the "result" to an integer or a string do? And how would you protect from mistakes, e.g. if you really wanted someAction() to return an int, but you accidentally used one that returns nothing?
Remember that the compiler has to generate code for all of the sites where you call someAction, so it needs to know what to do ahead of time, even if you may otherwise ensure that no "incorrect" calls are made.
You could implement a custom Action class on top of std::function that could support std::optional<ResultType> instead of ResultType. That way, if the actual result of the functor wrapped in Action was convertible to that ResultType, you'd get a non-null optional, otherwise the optional would be null. And the list of supported result types would need to be pre-determined, i.e. the compiler wouldn't be able to dynamically add them, but adding a new type should amount to passing an additional type as a template argument, so it'd be easy to support quite many types.
I have a C++ library that is heavily callback driven. The callbacks are registered as std::function instances.
A callback registration function might look something like:
void register(std::function<void(int)> callback);
I can register plain cdef functions by making libcpp.functional.function objects. Say I have this callback:
cdef void my_callback(int n):
# do something interesting with n
I can register it succesfully:
cdef function[void(int)]* f_my_callback = new function[void(int)](my_callback)
register(deref(f_my_callback))
The problem is that I want to register cdef class methods as the callback. Let's say I have a class:
cdef class MyClass:
cdef function[void(int)]* f_callback
py_callback = lambda x: None
# some more init/implementation
def __init__(self):
# ** this is where the problem is **
self.f_callback = new function[void(int)](self.member_callback)
register(deref(self.f_callback))
cdef void member_callback(self, int n) with gil:
self.py_callback(n)
def register_py(self, py_callback):
self.py_callback = py_callback
the self.f_callback = new function[void(int)](self.member_callback) line doesn't work because the function[T] constructor is seeing the MyClass parameter (self). In regular python, doing self.member_callback is basically equivalent to partially applying self to MyClass.member_callback, so I thought that this would be fine, but it is not.
I made a typedef:
ctypedef void (*member_type)(int)
Then if I cast, I can get it to compile:
self.f_callback = new function[void(int)](<member_type>self.member_callback)
But when callbacks actually come in, everything is broken. Doing anything with self results in a segfault.
What is the 'right' way to pass a cython class member method as a C/C++ callback?
EDIT:
Just to clarify, this question is specifically about passing member methods (i.e. function with self as the first argument) to C++ functions expecting a parameter of type std::function.
If I decorate the member_callback with #staticmethod and remove all references to self it all works as expected. Unfortunately, the method needs to have access to self to correctly execute the logic for the class instance.
std::function can take a range of arguments. In its simplest form it takes a function pointer that directly maps to its template type, and this is all that the Cython wrappers for it are really set up to cope with. For wrapping a C++ member function you'd typically use std::mem_fun or std::mem_fun_ref to either take a copy of, or a reference to, the relevant class instance (in modern C++ you might also choose a lambda function, but that really just gives you the same options).
Converting this to Python/Cython you need to hold a PyObject* to the object whose member you are calling, and with that comes a responsibility to handle the reference counting yourself. Unfortunately Cython can't currently generate the wrappers for you, and so you need to write a holder yourself. Essentially you need a callable object that handles the reference counting.
In a previous answer I showed two ways of creating this kind of wrapper (one of which was manual, and the second of which saved effort by using Boost Python, which has already implemented something very similar). The scheme I showed there will work for any Python callable matching the relevant signature. Although I illustrated it with a standard Python module level function it would work equally well for a member function since instance.memberfunction generates a bound member function - a Python callable which will work just as well.
For your problem the only difference is that you are using a cdef member function rather than a def member function. This looks like it should work but doesn't (recent versions of Cython attempt to do an automatic conversion to a Python callable but it fails with a runtime error). You can create a lambda function though as a very simple wrapper. This is slightly less-than-ideal from a speed point of view but has the advantage of working with the existing code.
The modifications you will need to make to earlier answer are as follows:
If you are trying to use the manual PyObjectWrapper version then change the argument types to match your signature (i.e. change int, string& to int). You don't have to do this for the boost version.
Your wrapper for void register(std::function<void(int)> callback); needs to be:
cdef extern from "whatever.hpp":
void register(PyObjWrapper)
# or
void register(bpo)
depending on which version you're using. This is lying to Cython about the signature, but since both of these objects are callable C++ objects they are automatically convertable to std::function by the C++ compiler.
Call register as register(PyObjWrapper(lambda x: self.member_callback(x))) or register(get_as_bpo(lambda x: self.member_callback(x))).
It would be possible to create a more efficient version specifically targeted at using cdef functions. It would be based around an object that is pretty similar to a PyObjWrapper. However, I'm reluctant to do this without a clear justification given that the code I've already written will work.
I believe that #DavidW's answer is correct for the simplified version of the problem that asked.
I presented a simplified version of my question in the interest of clarity, but so far as I can tell, the real version of my problem needs a slightly different approach.
Specifically, in my simplified version, I indicated that the callback would be called with an int. If this was the case, I could conceivably provide a python callable (either the boost python or the PyObjWrapper version).
However, the callbacks are actually called with a std::shared_ptr<T> where T is a templated class in the library. The cdef callback needs to do something with that instance and eventually call a python callable. So far as I can tell, there isn't a way to call a python function with a C++ object like a std::shared_ptr.
#DavidW's response was quite helpful though, and it led me to a solution that does work for this kind of situation. I made a C++ utility function:
// wrapper.hpp
#include <memory>
#include <functional>
#include "my_cpp_project.hpp"
template<typename T>
using cy_callback = void (*)(void*, std::shared_ptr<my_cpp_class<T>>);
template<typename T>
class function_wrapper {
public:
static
std::function<void(std::shared_ptr<my_cpp_class<T>>)>
make_std_function(cy_callback<T> callback, void* c_self)
{
std::function<void(std::shared_ptr<my_cpp_class<T>>)>
wrapper = [=](std::shared_ptr<my_cpp_class<T>> sample) -> void
{
callback(c_self, sample);
};
return wrapper;
}
};
Which I wrapped in cython:
cdef extern from "wrapper.hpp":
cdef cppclass function_wrapper[T]:
ctypedef void (*cy_callback) (void*, shared_ptr[my_cpp_class[T]])
#staticmethod
function[void(shared_ptr[my_cpp_class[T]])] make_std_function(
cy_callback, void*)
Basically, the idea is that I pass the cython class callback method to the wrapper and it returns a std::function version of it with the right type signature. The returned function is actually a lambda which applies the self argument to the callback.
Now I can make my cdef member methods and they get called correctly:
cdef void member_callback(self, shared_ptr[my_cpp_class[specific_type]] sample) with gil:
# do stuff with the sample ...
self.python_cb(sample_related_stuff)
registration looks something like:
register(function_wrapper[specific_type].make_std_function(<cb_type>self.member_callback, <void*>self))
Note the cb_type cast in there. The wrapper is expecting a void* so we need to cast it to match. This is the typedef:
ctypedef void (*cb_type)(void*, shared_ptr[my_cpp_class[specific_type]])
Hopefully this will be useful to someone in a similar boat. Thanks to #DavidW for getting me on the right track.
I'm doing a messaging library around websocket++ library. This library allows me to set my own functions for manage the messages. As I am not doing the final application that will use my code I need to allow also to my users to set their desired functions for this handlers.
This is how it works now. Somewhere there is the socket(m_client) class that has his set_handler function that is used like the following snippet.
m_client.set_message_handler(bind(&myFunction,&m_client,_1,_2));
what i would like to do is provide a function that will take as parameter &myFunction so the user just call like:
my_class.set_msg_handler(&myFunction);
and then this will be the declaration:
void set_msg_handler(<type> myfunction){
m_client.set_message_handler(bind(myFunction,&m_client,_1,_2));
}
But I could not make clear which is the correct type for myfunction in order to make it compile. Or if even this is a good approach to archive this. I went through boost code to see if i could get some hint... but as it ends working with templates, thats something that i don't manage yet.
I know it could be simpler to make the user itself do the bind and pass down its output, but m_client is not accessible directly and i would like to keep that way for encapsulation proposes. And I imagine that is not strictly necessary now, maybe some day I will need it anyway. So in the propose of learning i decided to ask it any way.
I'm quite new at C++ for this level of usage and the whole function pointers and handlers and templates scape a little to my actual understanding. I read about bind usage but all the examples consist on declaring a function and then using bind directly.
Also searched for a similar question, but didn't found it if it exist. And I will not discard that my approach is not the best or completely wrong, so all advises and recommendations will be welcome.
boost is using template
maybe you should try something like
template <typename T>
void set_msg_handler(T myfunction){
m_client.set_message_handler(bind(myFunction,&m_client,_1,_2));
}
But if you want to force the user to give some kind of function you can check for templates specializations.
But bind will accept almost any kind of pointers.
OK, finally I make it work as I was expecting. The solution was pretty simple, just a problem about my experience in c++.
First of all on the header of the class that will be performing the final bind I had declared a type declaring a function pointer with the structure that a message_handler must have. It will be easy to define the wrapper methods in oder classes:
typedef void (*msg_handler_ptr)(client*,connection_hdl,message_ptr);
It took me time to get the correct declaration... Once done, I have declared in the corresponding class the function that will be performint the binding:
void set_msg_handler(msg_handler_ptr myfunction){
using websocketpp::lib::placeholders::_1;
using websocketpp::lib::placeholders::_2;
using websocketpp::lib::bind;
m_client.set_message_handler(bind(myfunction,&m_client,::_1,::_2));
}
Last the wrapper for that on my controller class:
#include "mysocket_class.h"
void mycontroller::set_msg_handler(msg_handler_ptr myfunction)
{
c.set_msg_handler(myfunction);
}
And wherever I want I declare my real msg_handler that to be very simple it will only send to standard output what it recieves:
void on_message(client* c, websocketpp::connection_hdl hdl, message_ptr msg)
{
client::connection_ptr con = c->get_con_from_hdl(hdl);
std::cout << "Recived " << msg->get_payload() << std::endl;
}
int main(int argc, char* argv[]) {
mycontroller c();
c.set_msg_handler(&on_message);
}
And that's how you pass a function as a parameter of another function seems. And i have my app up an running.
Thanks you all for your hints!
I'm trying to learn how to best use the std::function and the std::bind facilities
in the standard library - I'm interested in the TR1 versions, as that's what I
have available for now and I'm not aware of the differences, if any, between the TR1 implementations and the C++11 ones.
So for the sake of the exercise I've set up to construct a simple let's say "dispatcher".
What I want is to be able to execute any function from the dispatcher based on some decisions taken later at runtime. I set up my class to have a general function data
member like so:
class PoorDispatcher
{
...
private:
std::tr1::function<void()> m_f;
}
Then I assign to the data member the function I really want to call, similar to the below
...
m_f = std::tr1::bind(some_func, param1, param2, param3);
...
// then the call
m_f(); // SUCCESS
The above allows me to call successfully the function I want but I'm not sure it's the right thing to do. The questions:
Is the above usage scenario "sane"? Are there any better alternatives?
The above method poses a problem when trying to bind to a function which returns something. How can I retrieve the return value? (In my silliness I tired to cast the function objects without much success)
The template argument to std::function is the actual function type. void() means a function which takes no arguments and returns no value.
If you want to store a function that returns a value you have to create a new function object. Or if you are not sure if the function will return something, use boost.optional.
How to simulate C# typeof-command behavior in C++?
C# example:
public static PluginNodeList GetPlugins (Type type)
{
...
}
Call:
PluginManager.GetPlugins (typeof(IPlugin))
How to implement this using C++? Maybe QT or Boost libraries provide a solution?
What about the case if you want to implement .GetPlugins(...) in a way that it loads those kinds of objects from a file (.so or .dll)?
You could use a dynamic_cast to test types as shown below:
IPlugin* iPluginPtr = NULL;
iPluginPtr = dynamic_cast<IPlugin*>(somePluginPtr);
if (iPluginPtr) {
// Cast succeeded
} else {
// Cast failed
}
This behaviour is called RTTI (Run time type information). This technique is best to be avoided, but can be beneficial in some situations.
There are two big ways to solve this. The first way is to write an interface with a pure virtual function that returns a class specific integer reference code. This code can then be used to represent a specific type. These integers could be stored in a specific enumeration.
In derived classes you can then override the method and return that class specific type.
During runtime, you can then call Plugin->getType() for instance, and it'll return its specific type. You can then perform a static_cast on the pointer to get the correct pointer of the derived type back.
The second way is to either use typeid to get the classtype of the object; but this is compiler dependant. You can also try casting your pointer using dynamic_cast; dynamic_cast returns a null pointer when it's being cast into the wrong type; and a valid one when being cast in a correct type. The dynamic cast method has a bigger overhead tho than the getType method described above.
If you want complete typeof-like behaviour, you would have to use RTTI (run-time type information). On many compilers you have to explicitly activate usage of RTTI, as it incurs run-time overhead.
Then you can use typeid or dynamic_cast to find an object's type.
If you don't want to use typeid, you'd have to use inheritance, pointers and/or overloads. Boost might help you, but it's not too hard.
Example 1:
class Iplugin { ... }
class Plugin1 : public Iplugin { ... }
class Plugin2 : public Iplugin { ... }
void getplugins(Iplugin* p) {
// ... you don't know the type, but you know
// what operations it supports via Iplugin
}
void getplugins(Plugin1& p) {
// expliticly handle Plugin1 objects
}
As you can see there are several ways of avoiding usage of RTTI and typeid.
Designing around this problem would be the best choice. Good use of object orientation can usually help but you can always create your own system for querying the type of an object by using a base class which stores an identifier for each object, for instance.
Always try to avoid using dynamic_cast as it most often uses string comparison to find the type of an object and that makes it really slow.
You can use typeof() in GCC. With other compilers, it's either not supported or you have to do crazy template mangling or use "bug-features" that are very compiler specific (like the way Boost does it).
Boost does have a typeof. C++ 0x doesn't call it typeof, but has both 'auto' and 'decltype' that provide the same kinds of functionality.
That said, I'm pretty sure none of those provides what you're really looking for in this case -- at most, they provide only a small piece of what you need/want overall.
Surely you would just use overloading?
static PluginManager::GetPlugins(Type1 &x) {
// Do something
}
static PluginManager::GetPlugins(Type2 &x) {
// Do something else
}
and then call:
PluginManager::GetPlugins(IPlugin);
Not directly answering the "how to get typeof() in C++", but I infer from your question that you are looking at how to do plugins in C++. If that's the case, you may be interested in the (not-yet)Boost.Extension library, and maybe in its reflection part.