Wrapping the observer pattern with Cython - c++

Question

The observer pattern appears frequently in my C++ project, which I now want to expose to the Python interpreter via Cython bindings. I tried to construct a minimal example illustrating the situation. A Spectacle accepts any object derived from the abstract base class Observer, such as an Onlooker. When we call Spectacle::event(), each registered observer is notified.
This is the content of the file ObserverPattern.h:
class Spectacle {
private:
std::vector<Observer*> observers;
public:
Spectacle() {};
virtual ~Spectacle() {};
virtual void registerObserver(Observer* observer) {
this->observers.push_back(observer);
}
virtual void event() {
std::cout << "event triggered" << std::endl;
for (Observer* observer : this->observers) {
observer->onEvent();
}
}
};
class Observer {
public:
Observer() {};
virtual ~Observer() {};
virtual void onEvent() = 0;
};
class Onlooker : public Observer {
public:
Onlooker() {};
virtual ~Onlooker() {};
virtual void onEvent() {
std::cout << "event observed" << std::endl;
}
};
And this is the content of my .pyx file, containing the bindings:
cdef extern from "ObserverPattern.h":
cdef cppclass _Spectacle "Spectacle":
_Spectacle() except +
void registerObserver(_Observer* observer)
void event()
cdef extern from "ObserverPattern.h":
cdef cppclass _Observer "Observer":
_Observer() except +
void onEvent()
cdef extern from "ObserverPattern.h":
cdef cppclass _Onlooker "Onlooker":
_Onlooker() except +
void onEvent()
cdef class Spectacle:
cdef _Spectacle _this
def event(self):
self._this.event()
def registerObserver(self, Observer observer):
self._this.registerObserver(observer._this)
cdef class Observer:
cdef _Observer* _this # must be a pointer because _Observer has pure virtual method
cdef class Onlooker(Observer):
pass # what should be the class body?
This does compile, but segfaults when event() is called and the observers are notified:
>>> spec = CythonMinimal.Spectacle()
>>> look = CythonMinimal.Onlooker()
>>> spec.registerObserver(look)
>>> spec.event()
event triggered
Segmentation fault: 11
What is the problem here and how could a fix look like?

Answer 1

Your problem is essentially "implement a C++ interface in Python".
The only portable way to do this is to write an actual C++ class
that will call back into Python.
Cython has undocumented experimental_cpp_class_def option that allows to
create C++ classes using Cython syntax. It's not pretty (IMO), but it works
for many scenarios.
Here is how you could implement Observer that delegates to the provided
Python callable:
from cpython.ref cimport PyObject, Py_INCREF, Py_DECREF
cdef cppclass ObserverImpl(_Observer):
PyObject* callback
__init__(object callback): # constructor. "this" argument is implicit.
Py_INCREF(callback)
this.callback = <PyObject*>callback
__dealloc__(): # destructor
Py_DECREF(<object>this.callback)
void onEvent():
(<object>this.callback)() # exceptions will be ignored
And that's how you could use it:
def registerObserver(self, callback not None): # user passes any Python callable
self._this.registerObserver(new ObserverImpl(callback))
C++ objects, just like C structures, can't hold Cython-managed object
references. That's why you have to use PyObject* field and manage reference
counting yourself. Inside methods you can, of course, cast to and use any Cython feature.
Another tricky moment is exception propagation. onEvent() method, being defined in C++, can't propagate Python exceptions. Cython will simply ignore exceptions it can't propagate. If you want to do better, catch them yourself and store somewhere for later inspection or rethrow as C++ exception. (I think it's impossible to throw C++ exceptions in Cython syntax, but you can call an external throwing helper function.)
If your observer has more than one method, then callback would be a Python class and instead of calling it directly you would call its methods, like (<object>this.callback).onEvent().
Obviously, ObserverImpl can also be coded directly in C++. Py_INCREF, Py_DECREF and PyObject_Call/PyObject_CallMethod are the only Python APIs necessary.