I am trying to achieve call Python functions from C++. I thought it could be achieved through function pointers, but it does not seem to be possible. I have been using boost.python to accomplish this.
Say there is a function defined in Python:
def callback(arg1, arg2):
#do something
return something
Now I need to pass this function to C++, so that it can be called from there. How do I write the code on C++ side using boost.python to achieve this?
If it might have any name:
Pass it to a function that takes a boost::python::object.
bp::object pycb; //global variable. could also store it in a map, etc
void register_callback(bp::object cb)
{
pycb = cb;
}
If it is in a single known namespace with a consistent name:
bp::object pycb = bp::scope("namespace").attr("callback");
bp::object has operator() defined, so you call it just like any function
ret = pycb()
Not a clue. But you can use PyObject_Call() to call it once you have the function object.
I've not used it before, but the reference manual has a section called Calling Python Functions and Methods which seems to show how to do this.
I used PyRun_SimpleString("myFunction()") as quick hack, as my function's name was known, took no args and lived in the __main__ namespace. Note you additionally need to get lock GIL if you are multi-threaded.
Related
I have a C++ library that is heavily callback driven. The callbacks are registered as std::function instances.
A callback registration function might look something like:
void register(std::function<void(int)> callback);
I can register plain cdef functions by making libcpp.functional.function objects. Say I have this callback:
cdef void my_callback(int n):
# do something interesting with n
I can register it succesfully:
cdef function[void(int)]* f_my_callback = new function[void(int)](my_callback)
register(deref(f_my_callback))
The problem is that I want to register cdef class methods as the callback. Let's say I have a class:
cdef class MyClass:
cdef function[void(int)]* f_callback
py_callback = lambda x: None
# some more init/implementation
def __init__(self):
# ** this is where the problem is **
self.f_callback = new function[void(int)](self.member_callback)
register(deref(self.f_callback))
cdef void member_callback(self, int n) with gil:
self.py_callback(n)
def register_py(self, py_callback):
self.py_callback = py_callback
the self.f_callback = new function[void(int)](self.member_callback) line doesn't work because the function[T] constructor is seeing the MyClass parameter (self). In regular python, doing self.member_callback is basically equivalent to partially applying self to MyClass.member_callback, so I thought that this would be fine, but it is not.
I made a typedef:
ctypedef void (*member_type)(int)
Then if I cast, I can get it to compile:
self.f_callback = new function[void(int)](<member_type>self.member_callback)
But when callbacks actually come in, everything is broken. Doing anything with self results in a segfault.
What is the 'right' way to pass a cython class member method as a C/C++ callback?
EDIT:
Just to clarify, this question is specifically about passing member methods (i.e. function with self as the first argument) to C++ functions expecting a parameter of type std::function.
If I decorate the member_callback with #staticmethod and remove all references to self it all works as expected. Unfortunately, the method needs to have access to self to correctly execute the logic for the class instance.
std::function can take a range of arguments. In its simplest form it takes a function pointer that directly maps to its template type, and this is all that the Cython wrappers for it are really set up to cope with. For wrapping a C++ member function you'd typically use std::mem_fun or std::mem_fun_ref to either take a copy of, or a reference to, the relevant class instance (in modern C++ you might also choose a lambda function, but that really just gives you the same options).
Converting this to Python/Cython you need to hold a PyObject* to the object whose member you are calling, and with that comes a responsibility to handle the reference counting yourself. Unfortunately Cython can't currently generate the wrappers for you, and so you need to write a holder yourself. Essentially you need a callable object that handles the reference counting.
In a previous answer I showed two ways of creating this kind of wrapper (one of which was manual, and the second of which saved effort by using Boost Python, which has already implemented something very similar). The scheme I showed there will work for any Python callable matching the relevant signature. Although I illustrated it with a standard Python module level function it would work equally well for a member function since instance.memberfunction generates a bound member function - a Python callable which will work just as well.
For your problem the only difference is that you are using a cdef member function rather than a def member function. This looks like it should work but doesn't (recent versions of Cython attempt to do an automatic conversion to a Python callable but it fails with a runtime error). You can create a lambda function though as a very simple wrapper. This is slightly less-than-ideal from a speed point of view but has the advantage of working with the existing code.
The modifications you will need to make to earlier answer are as follows:
If you are trying to use the manual PyObjectWrapper version then change the argument types to match your signature (i.e. change int, string& to int). You don't have to do this for the boost version.
Your wrapper for void register(std::function<void(int)> callback); needs to be:
cdef extern from "whatever.hpp":
void register(PyObjWrapper)
# or
void register(bpo)
depending on which version you're using. This is lying to Cython about the signature, but since both of these objects are callable C++ objects they are automatically convertable to std::function by the C++ compiler.
Call register as register(PyObjWrapper(lambda x: self.member_callback(x))) or register(get_as_bpo(lambda x: self.member_callback(x))).
It would be possible to create a more efficient version specifically targeted at using cdef functions. It would be based around an object that is pretty similar to a PyObjWrapper. However, I'm reluctant to do this without a clear justification given that the code I've already written will work.
I believe that #DavidW's answer is correct for the simplified version of the problem that asked.
I presented a simplified version of my question in the interest of clarity, but so far as I can tell, the real version of my problem needs a slightly different approach.
Specifically, in my simplified version, I indicated that the callback would be called with an int. If this was the case, I could conceivably provide a python callable (either the boost python or the PyObjWrapper version).
However, the callbacks are actually called with a std::shared_ptr<T> where T is a templated class in the library. The cdef callback needs to do something with that instance and eventually call a python callable. So far as I can tell, there isn't a way to call a python function with a C++ object like a std::shared_ptr.
#DavidW's response was quite helpful though, and it led me to a solution that does work for this kind of situation. I made a C++ utility function:
// wrapper.hpp
#include <memory>
#include <functional>
#include "my_cpp_project.hpp"
template<typename T>
using cy_callback = void (*)(void*, std::shared_ptr<my_cpp_class<T>>);
template<typename T>
class function_wrapper {
public:
static
std::function<void(std::shared_ptr<my_cpp_class<T>>)>
make_std_function(cy_callback<T> callback, void* c_self)
{
std::function<void(std::shared_ptr<my_cpp_class<T>>)>
wrapper = [=](std::shared_ptr<my_cpp_class<T>> sample) -> void
{
callback(c_self, sample);
};
return wrapper;
}
};
Which I wrapped in cython:
cdef extern from "wrapper.hpp":
cdef cppclass function_wrapper[T]:
ctypedef void (*cy_callback) (void*, shared_ptr[my_cpp_class[T]])
#staticmethod
function[void(shared_ptr[my_cpp_class[T]])] make_std_function(
cy_callback, void*)
Basically, the idea is that I pass the cython class callback method to the wrapper and it returns a std::function version of it with the right type signature. The returned function is actually a lambda which applies the self argument to the callback.
Now I can make my cdef member methods and they get called correctly:
cdef void member_callback(self, shared_ptr[my_cpp_class[specific_type]] sample) with gil:
# do stuff with the sample ...
self.python_cb(sample_related_stuff)
registration looks something like:
register(function_wrapper[specific_type].make_std_function(<cb_type>self.member_callback, <void*>self))
Note the cb_type cast in there. The wrapper is expecting a void* so we need to cast it to match. This is the typedef:
ctypedef void (*cb_type)(void*, shared_ptr[my_cpp_class[specific_type]])
Hopefully this will be useful to someone in a similar boat. Thanks to #DavidW for getting me on the right track.
I am using Lua to write scipts and embed them in C++. I use LuaBridge in this process.
In my Lua script, I have some variables which need to be retrieved at first for usage in C++, besides, I have a very simple function:
run = function()
print ("state is on!")
end
This function, however, is only called under a specific condition. i.e. only called when a "true" is obtained from C++ codes after a series of complicated calculations.
Limited by my Lua and LuaBridge knowledge, what I know is: after I do
loadscript(L, "script.lua")
lua_pcall(L,0,0,0)
I can read out variables and functions from Lua script by using
LuaRef blabla = getGlobal(L, "blabla")
But now, I need to read out variables and use them at first, then in a member function
LuaRunner::LuaRun(){}
defined in a separate C++ class
class LuaRunner
the condition will be obatined and this run() function will be called if the condition is "true". It would be best to call this run() function right in the C++ member function
LuaRunner::LuaRun(){}
due to restriction of further processing.
Therefore, I was wondering whether this would be possible:
Read out the function using
LuaRef run = getGlobal(L, "run")
together with other variables at the beginning and "save" this run() function somewhere in C++ codes (perhaps as a class member function), and then the run() function can be called later on by a pointer or an object in the same class. Would this be possible? If possible, how to do it? Or any other good ideas?
It's possible to store luabridge::LuaRef's in C++ to call them later just as you normally call any other function. Though sometimes there's no need to store LuaRef's anywhere. Once you load the script, all functions stay in your lua_State, unless you set them to nil or override them by loading another script which uses the same names for functions. You can get them by using getGlobal function. If your Lua function takes arguments, you can pass them, using LuaRef's operator() like this:
if(...) { // your call condition
LuaRef f = getGlobal(L, "someFunction");
f(arg1, arg2, ...); // if your function takes no arguments, just use f();
}
I'm trying to learn how to best use the std::function and the std::bind facilities
in the standard library - I'm interested in the TR1 versions, as that's what I
have available for now and I'm not aware of the differences, if any, between the TR1 implementations and the C++11 ones.
So for the sake of the exercise I've set up to construct a simple let's say "dispatcher".
What I want is to be able to execute any function from the dispatcher based on some decisions taken later at runtime. I set up my class to have a general function data
member like so:
class PoorDispatcher
{
...
private:
std::tr1::function<void()> m_f;
}
Then I assign to the data member the function I really want to call, similar to the below
...
m_f = std::tr1::bind(some_func, param1, param2, param3);
...
// then the call
m_f(); // SUCCESS
The above allows me to call successfully the function I want but I'm not sure it's the right thing to do. The questions:
Is the above usage scenario "sane"? Are there any better alternatives?
The above method poses a problem when trying to bind to a function which returns something. How can I retrieve the return value? (In my silliness I tired to cast the function objects without much success)
The template argument to std::function is the actual function type. void() means a function which takes no arguments and returns no value.
If you want to store a function that returns a value you have to create a new function object. Or if you are not sure if the function will return something, use boost.optional.
I would like to make an alias in C++ to singleton calling
so instead of calling MYCLASS::GetInstance()->someFunction(); each time, I could call just someFunctionAlias(); in my code.
Use a static function.
namespace ... {
void someFunction() {
MYCLASS::GetInstance()->someFunction();
}
};
Edit: Sorry lads, I wrote static someFunction and meant void someFunction.
typedefs are used for type aliases but can't be used as call alias.
functions (such as suggested as by DeadMG) can be used as a call "alias".
PS. As this is C++ you have lots of options, function pointers, std::tr1::function<> operator overloading and the preprocessor. But in this case it certainly looks like a simple function would be the simplest and best solution.
Look up function pointers.
You can create a function pointer, and assign it to your long function. You can then call this function pointer just like a regular function, wherever your variable is defined.
Function pointers can be confusing, but are used a lot in API callbacks (i.e. you pass a function as an argument to the API, and the API will call that function when something happens (think WndProc)).
Good luck.
you can do this
#define someFunctionAlias MYCLASS::GetInstance()->someFunction()
I have a DLL which has a function which accepts a function pointer converts it to a boost::function. This is then stored and then called from inside the DLL.
I want to add a function to the DLL to handle member functions in a similar way. I know I need to use boost::bind to wrap the member function pointer and the object together. I want to do the binding on the DLL side though so the EXE does not require boost libraries to be included.
How would you write this function? One which accepts a member function pointer and an object as arguments and binds them together.
Thanks!
you're warned by #Kylotan, so you can try something like this:
__declspec(dllexport) void store_mem_fn(void(Your_class::*mem_fn)(void), Your_class& instance)
{
std::vector<boost::function<void(void)> > container;
container.push_back(boost::bind(mem_fn, instance));
}
It might be a bad idea to try passing member function pointers into DLLs because they can vary in size depending on certain circumstances. (Some details here.) Maybe if you always know that you will be building both halves of the application with the same compiler you will be ok.
As for the function, I expect it would look something like this (completely untested and uncompiled code):
typedef void(ObjectType::*OTMemberFn)();
boost::function<void (ObjectType o)> bind_mem_fn(ObjectType o, OTMemberFn mf)
{
return boost::bind(mf, o);
}
Isn't Boost open source? If so, peek into the boost code, learn how it's done, and re-implement it yourself, without the dependency.