I am currently using SWIG to make the implementation of small modules easier in my main C++ programm. The class architecture is as follow :
foo.hpp :
class Foo
{
public:
virtual int pureFunc() = 0;
int func() { return (42); }
};
file.i :
%module(directors="1") foo
%feature("director") Foo;
%{
#include "foo.hpp"
%}
%include "foo.hpp"
file.py :
import sys
sys.path.append('.')
import foo
class Bar(foo.Foo):
def __init__(self):
pass
def pureFunc(self):
return 21
lol = Bar()
print lol.pureFunc()
print lol.func()
I then generate the swig wrappers using the following command :
swig -python -c++ file.i
and compile the .so like this :
g++ -fPIC -shared -o _foo.so file_wrap.cxx -I/usr/include/python2.7 -lpython2.7
And when I try to run the python script I get the following error :
# python file.py
21
Traceback (most recent call last):
File "file.py", line 13, in <module>
print lol.func()
File "/home/volent/dev/CPP/cython/foo.py", line 84, in func
def func(self): return _foo.Foo_func(self)
TypeError: in method 'Foo_func', argument 1 of type 'Foo *'
# _
That shows that the use of the pure method is working but I can't use the one already defined in the .hpp file.
I have tried to read the SWIG documentation and the only things I see about abstract class and inheritance are 34.4.7 C++ Classes and 34.4.8 C++ Inheritance. And I can't see anything mentionning a case like that.
You have forgotten to call the __init__ method of the parent class of Bar. Replace your __init__ method with this:
class Bar(foo.Foo):
def __init__(self):
super(Bar,self).__init__()
This should let your Bar class know about the Foo method.
Related
So I am writing a Python API for a C++ library using Cython. I have three classes with almost identical functionality: A, B, and C. This difference is only how one of their objects is built on initialization and some constants.
Originally, I have wrote them all out as separate classes and then could define them via extern in my cython code. This compiles and works well but there is a lot of repeated code and I would really like this project to be more DRY.
So I decided to write a base class for A, B, and C that implemented most of the functionality once. However, I needed to template that base class and this is causing me a nightmare when I try to define everything in Cython. Here is a toy example of what I'm talking about (ignore missed semi-colons etc, if you find them). This is my "classes.h" file
int library_method_load(std::string file_name){
return std::string.length();
}
template <class T>
class BaseClass{
public:
T important_obj;
BaseClass(std::string file_name){ important_obj = library_method(file_name);};
virtual T library_method(std::string file_name) = 0;
// Important logicks...
~BaseClass(){};
}
class A : public BaseClass<int> {
A(std::string file_name): BaseClass<int>(file_name){};
int library_method(std::string file_name){ return library_method_load(file_name);};
~A(){};
}
When I try to wrap this, if I don't tell cython about the base class, I get undefined symbols. If I try to define the base class, the templating causes problems. The latter could be due to the fact that I don't know the syntax properly for inheriting templated base classes.
Here is my current attempt
#distutils: language = c++
from libcpp.string cimport string
cdef extern from "classes.h":
cppclass BaseClass[T]:
BaseClass(string file_name)
cdef extern from "classes.h":
cppclass A(BaseClass[int]):
A(string file_name)
cdef class PyBase:
cdef BaseClass* wrapped
cdef class PyA(PyBase):
def __cinit__(self, string file_name):
self.wrapped = <BaseClass[int]*> new A(file_name)
Doing this gives me the following compiler error:
Error compiling Cython file:
------------------------------------------------------------ ...
cdef class PyA(PyBase):
def __cinit__(self, string file_name):
self.wrapped = <BaseClass[int]*> new A(file_name)
^
wrapper.pyx:23:23: Cannot assign type 'BaseClass[int] *' to
'BaseClass[T] *' Traceback (most recent call last): File "setup.py",
line 9, in
setup(name="test", version="1.0.0", ext_modules=cythonize([rk])) File
"/home/jacob/anaconda3/lib/python3.6/site-packages/Cython/Build/Dependencies.py",
line 1027, in cythonize
cythonize_one(*args) File "/home/jacob/anaconda3/lib/python3.6/site-packages/Cython/Build/Dependencies.py",
line 1149, in cythonize_one
raise CompileError(None, pyx_file) Cython.Compiler.Errors.CompileError: wrapper.pyx
Does anyone know how to do this? Clearly my template substituion is off. Should I stick with the repetitive code instead? Are there other clever solutions?
One solution is to drop PyBase because in cdef BaseClass* wrapped the template argument for BaseClass is missing, which makes the line meaningless.
E.g.:
cdef class PyA(PyBase):
cdef A* wrapped
def __cinit__(self, string file_name):
self.wrapped = new A(file_name)
(I am not sure though if you can pass a C++ object into __init__ and __cinit__)
I use Cython 0.19.2 (and Python 2.7.1) to expose C++ classes to Python.
As a first try, i did a test with the 'Rectangle' class example of the documentation.
http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html
I have a crash which I don't understand.
I have tried to simplify the code the the max. but I still have the problem.
Here is my pyx file, the C++ sources are just a cut&paste from the python/cython's documentation.
# distutils: language = c++
# distutils: sources = Rectangle.cpp
cdef extern from "Rectangle.h" namespace "shapes":
cdef cppclass Rectangle:
pass
cdef class PyRectangle:
cdef Rectangle* thisptr
I just want to declare a class with a thisptr, which points the the C++ Rectangle class instance.
When i try to compile the program with:
cython -a --cplus rect.pyx
I have the following crash:
Error compiling Cython file:
------------------------------------------------------------
...
cdef extern from "Rectangle.h" namespace "shapes":
cdef cppclass Rectangle:
pass
cdef class PyRectangle:
cdef Rectangle* thisptr
^
------------------------------------------------------------
rect.pyx:9:7: Compiler crash in AnalyseDeclarationsTransform
File 'ModuleNode.py', line 101, in analyse_declarations: ModuleNode(rect.pyx:1:0,
full_module_name = 'rect')
File 'Nodes.py', line 382, in analyse_declarations: StatListNode(rect.pyx:4:0)
File 'Nodes.py', line 4251, in analyse_declarations: CClassDefNode(rect.pyx:8:5,
as_name = u'PyRectangle',
class_name = u'PyRectangle',
module_name = u'',
visibility = u'private')
File 'Nodes.py', line 382, in analyse_declarations: StatListNode(rect.pyx:9:7)
File 'Nodes.py', line 1208, in analyse_declarations: CVarDefNode(rect.pyx:9:7,
modifiers = [...]/0,
visibility = u'private')
Compiler crash traceback from this point on:
File "/home/xxx/local/python2.7.1/site-packages/Cython/Compiler/Nodes.py", line 1208, in analyse_declarations
self.entry.doc = embed_position(self.pos, self.doc)
AttributeError: 'CVarDefNode' object has no attribute 'doc'
I have tried to compile with pyrex, setup.py, ... everything. But I still have the same error.
Is there something I'm missing?
Thanks
Alright, I finally fixed it. My version of Python 2.7.1 simply doesn't work with the last Cython 0.19.2. I upgraded to 2.7.6 and it works.
I am no stranger to the python ctypes module, but this is my first attempt at combining C++, C and Python all in one code. My problem seems to be very similar to Seg fault when using ctypes with Python and C++, however I could not seem to solve the problem in the same way.
I have a simple C++ file called Header.cpp:
#include <iostream>
class Foo{
public:
int nbits;
Foo(int nb){nbits = nb;}
void bar(){ std::cout << nbits << std::endl; }
};
extern "C" {
Foo *Foo_new(int nbits){ return new Foo(nbits); }
void Foo_bar(Foo *foo){ foo->bar(); }
}
which I compile to a shared library using:
g++ -c Header.cpp -fPIC -o Header.o
g++ -shared -fPIC -o libHeader.so Header.o
and a simple Python wrapper called test.py:
import ctypes as C
lib = C.CDLL('./libHeader.so')
class Foo(object):
def __init__(self,nbits):
self.nbits = C.c_int(nbits)
self.obj = lib.Foo_new(self.nbits)
def bar(self):
lib.Foo_bar(self.obj)
def main():
f = Foo(32)
f.bar()
if __name__ == "__main__":
main()
I would expect that when I call test.py, I should get the number 32 printed to screen. However, all I get is a segmentation fault. If I change the constructor to return the class instance on the stack (i.e. without the new call) and then pass around the object, the program performs as expected. Also, if I change the bar method in the Foo class such that it does not use the nbits member, the program does not seg fault.
I have an limited understanding of C++, but the fact that I can make this function as expected in C and in C++ but not in Python is a little confusing. Any help would be greatly appreciated.
Update: Thanks to one of the comments below, the problem has been solved. In this case, an explicit declaration of both restype and argtypes for the C functions was required. i.e the following was added to the python code:
lib.Foo_new.restype = C.c_void_p
lib.Foo_new.argtypes = [C.c_int32]
lib.Foo_bar.restype = None
lib.Foo_bar.argtypes = [C.c_void_p]
I would try the following:
extern "C"
{
Foo *Foo_new(int nbits)
{
Foo *foo = new Foo(nbits);
printf("Foo_new(%d) => foo=%p\n", nbits, foo);
return foo;
}
void Foo_bar(Foo *foo)
{
printf("Foo_bar => foo=%p\n", foo);
foo->bar();
}
}
to see if the values of foo match.
Also, you might want to look at Boost.Python to simplify creating Python bindings of C++ objects.
I build a .so C++ library using g++ and -fPIC (using eclipse).
Still using eclipse, I linked this library and used it in another C++ project without any problem.
But,
When I build a Cython project with that same lib to generate a python extension, using :
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [
Extension("cyelp",
sources=["cyelp.pyx", \
"adapter/ATestClass.cpp", \
"adapter/ALabSimulatorTime.cpp", \
],
libraries=["elp"],
language="c++",
)
]
)
"libelp.so" being the mentioned library, the build is fine too : I get my cyelp.so library.
The problem occurs when I get a specific class from the library at runtime from python side script :
Here is my cython class (that inherits from a ALabSimulationTime:LabSimulationTime class implementing the method FireEvent() - method which is declared as "pure virtual" in LabSimulationTime) :
cimport cpython.ref as cpy_ref
cdef extern from "adapter/ALabSimulatorTime.h" namespace "elps" :
cdef cppclass ALabSimulatorTime:
ALabSimulatorTime(cpy_ref.PyObject *obj)
# Virtual overridable
void ResetTime()
double TimeStep()
void FireEvent()
void StepSimulation()
int EndSimulation()
void RunSimulation()
# Others
void UpdateEventsRate(double rate)
void SetEndTime(double end_time)
void SetOutputTimeStep(double out_time_step)
double GetTime()
int GetNbFiredEvents()
void SetTime(double time)
cdef class PyLabSimulatorTime:
cdef ALabSimulatorTime* thisptr
def __cinit__(self):
self.thisptr = new ALabSimulatorTime(<cpy_ref.PyObject*>self)
def __dealloc__(self):
if self.thisptr:
del self.thisptr
cpdef ResetTime(self):
self.thisptr.ResetTime()
cpdef double TimeStep(self):
return self.thisptr.TimeStep()
And here, my python loading attempt :
from cyelp import PyLabSimulatorTime;
Finally, here's the error message :
Traceback (most recent call last):
File "src/Spacial/BdmLsim2.py", line 1, in <module>
from cyelp import PyLabSimulatorTime;
ImportError: setup/cyelp.so: undefined symbol: _ZN4elps16LabSimulatorTime9FireEventEv
The fact is that it doesn't happen if I redefine the "FireEvent()" method in ALabSimulatorTime class from the header file :
virtual void FireEvent() {};
But does happen if I redefine the method from the ".cpp" file :
void ALabSimulatorTime::FireEvent()
{
//...
}
Note : Everything works well if I turn FireEvent to "non-pure" from the base class "LabSimulatorTime".
I could, of course, try to be more specific, but may be some of you already has an idea about what is going on.
Thanks a lot
I have a class interface written in C++. I have a few classes that implement this interface also written in C++. These are called in the context of a larger C++ program, which essentially implements "main". I want to be able to write implementations of this interface in Python, and allow them to be used in the context of the larger C++ program, as if they had been just written in C++.
There's been a lot written about interfacing python and C++ but I cannot quite figure out how to do what I want. The closest I can find is here: http://www.cs.brown.edu/~jwicks/boost/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_virtual_functions, but this isn't quite right.
To be more concrete, suppose I have an existing C++ interface defined something like:
// myif.h
class myif {
public:
virtual float myfunc(float a);
};
What I want to be able to do is something like:
// mycl.py
... some magic python stuff ...
class MyCl(myif):
def myfunc(a):
return a*2
Then, back in my C++ code, I want to be able to say something like:
// mymain.cc
void main(...) {
... some magic c++ stuff ...
myif c = MyCl(); // get the python class
cout << c.myfunc(5) << endl; // should print 10
}
I hope this is sufficiently clear ;)
There's two parts to this answer. First you need to expose your interface in Python in a way which allows Python implementations to override parts of it at will. Then you need to show your C++ program (in main how to call Python.
Exposing the existing interface to Python:
The first part is pretty easy to do with SWIG. I modified your example scenario slightly to fix a few issues and added an extra function for testing:
// myif.h
class myif {
public:
virtual float myfunc(float a) = 0;
};
inline void runCode(myif *inst) {
std::cout << inst->myfunc(5) << std::endl;
}
For now I'll look at the problem without embedding Python in your application, i.e. you start excetion in Python, not in int main() in C++. It's fairly straightforward to add that later though.
First up is getting cross-language polymorphism working:
%module(directors="1") module
// We need to include myif.h in the SWIG generated C++ file
%{
#include <iostream>
#include "myif.h"
%}
// Enable cross-language polymorphism in the SWIG wrapper.
// It's pretty slow so not enable by default
%feature("director") myif;
// Tell swig to wrap everything in myif.h
%include "myif.h"
To do that we've enabled SWIG's director feature globally and specifically for our interface. The rest of it is pretty standard SWIG though.
I wrote a test Python implementation:
import module
class MyCl(module.myif):
def __init__(self):
module.myif.__init__(self)
def myfunc(self,a):
return a*2.0
cl = MyCl()
print cl.myfunc(100.0)
module.runCode(cl)
With that I was then able to compile and run this:
swig -python -c++ -Wall myif.i
g++ -Wall -Wextra -shared -o _module.so myif_wrap.cxx -I/usr/include/python2.7 -lpython2.7
python mycl.py
200.0
10
Exactly what you'd hope to see from that test.
Embedding the Python in the application:
Next up we need to implement a real version of your mymain.cc. I've put together a sketch of what it might look like:
#include <iostream>
#include "myif.h"
#include <Python.h>
int main()
{
Py_Initialize();
const double input = 5.0;
PyObject *main = PyImport_AddModule("__main__");
PyObject *dict = PyModule_GetDict(main);
PySys_SetPath(".");
PyObject *module = PyImport_Import(PyString_FromString("mycl"));
PyModule_AddObject(main, "mycl", module);
PyObject *instance = PyRun_String("mycl.MyCl()", Py_eval_input, dict, dict);
PyObject *result = PyObject_CallMethod(instance, "myfunc", (char *)"(O)" ,PyFloat_FromDouble(input));
PyObject *error = PyErr_Occurred();
if (error) {
std::cerr << "Error occured in PyRun_String" << std::endl;
PyErr_Print();
}
double ret = PyFloat_AsDouble(result);
std::cout << ret << std::endl;
Py_Finalize();
return 0;
}
It's basically just standard embedding Python in another application. It works and gives exactly what you'd hope to see also:
g++ -Wall -Wextra -I/usr/include/python2.7 main.cc -o main -lpython2.7
./main
200.0
10
10
The final piece of the puzzle is being able to convert the PyObject* that you get from creating the instance in Python into a myif *. SWIG again makes this reasonably straightforward.
First we need to ask SWIG to expose its runtime in a headerfile for us. We do this with an extra call to SWIG:
swig -Wall -c++ -python -external-runtime runtime.h
Next we need to re-compile our SWIG module, explicitly giving the table of types SWIG knows about a name so we can look it up from within our main.cc. We recompile the .so using:
g++ -DSWIG_TYPE_TABLE=myif -Wall -Wextra -shared -o _module.so myif_wrap.cxx -I/usr/include/python2.7 -lpython2.7
Then we add a helper function for converting the PyObject* to myif* in our main.cc:
#include "runtime.h"
// runtime.h was generated by SWIG for us with the second call we made
myif *python2interface(PyObject *obj) {
void *argp1 = 0;
swig_type_info * pTypeInfo = SWIG_TypeQuery("myif *");
const int res = SWIG_ConvertPtr(obj, &argp1,pTypeInfo, 0);
if (!SWIG_IsOK(res)) {
abort();
}
return reinterpret_cast<myif*>(argp1);
}
Now this is in place we can use it from within main():
int main()
{
Py_Initialize();
const double input = 5.5;
PySys_SetPath(".");
PyObject *module = PyImport_ImportModule("mycl");
PyObject *cls = PyObject_GetAttrString(module, "MyCl");
PyObject *instance = PyObject_CallFunctionObjArgs(cls, NULL);
myif *inst = python2interface(instance);
std::cout << inst->myfunc(input) << std::endl;
Py_XDECREF(instance);
Py_XDECREF(cls);
Py_Finalize();
return 0;
}
Finally we have to compile main.cc with -DSWIG_TYPE_TABLE=myif and this gives:
./main
11
Minimal example; note that it is complicated by the fact that Base is not pure virtual. There we go:
baz.cpp:
#include<string>
#include<boost/python.hpp>
using std::string;
namespace py=boost::python;
struct Base{
virtual string foo() const { return "Base.foo"; }
// fooBase is non-virtual, calling it from anywhere (c++ or python)
// will go through c++ dispatch
string fooBase() const { return foo(); }
};
struct BaseWrapper: Base, py::wrapper<Base>{
string foo() const{
// if Base were abstract (non-instantiable in python), then
// there would be only this->get_override("foo")() here
//
// if called on a class which overrides foo in python
if(this->get_override("foo")) return this->get_override("foo")();
// no override in python; happens if Base(Wrapper) is instantiated directly
else return Base::foo();
}
};
BOOST_PYTHON_MODULE(baz){
py::class_<BaseWrapper,boost::noncopyable>("Base")
.def("foo",&Base::foo)
.def("fooBase",&Base::fooBase)
;
}
bar.py
import sys
sys.path.append('.')
import baz
class PyDerived(baz.Base):
def foo(self): return 'PyDerived.foo'
base=baz.Base()
der=PyDerived()
print base.foo(), base.fooBase()
print der.foo(), der.fooBase()
Makefile
default:
g++ -shared -fPIC -o baz.so baz.cpp -lboost_python `pkg-config python --cflags`
And the result is:
Base.foo Base.foo
PyDerived.foo PyDerived.foo
where you can see how fooBase() (the non-virtual c++ function) calls virtual foo(), which resolves to the override regardless whether in c++ or python. You could derive a class from Base in c++ and it would work just the same.
EDIT (extracting c++ object):
PyObject* obj; // given
py::object pyObj(obj); // wrap as boost::python object (cheap)
py::extract<Base> ex(pyObj);
if(ex.check()){ // types are compatible
Base& b=ex(); // get the wrapped object
// ...
} else {
// error
}
// shorter, thrwos when conversion not possible
Base &b=py::extract<Base>(py::object(obj))();
Construct py::object from PyObject* and use py::extract to query whether the python object matches what you are trying to extract: PyObject* obj; py::extract<Base> extractor(py::object(obj)); if(!extractor.check()) /* error */; Base& b=extractor();
Quoting http://wiki.python.org/moin/boost.python/Inheritance
"Boost.Python also allows us to represent C++ inheritance relationships so that wrapped derived classes may be passed where values, pointers, or references to a base class are expected as arguments."
There are examples of virtual functions so that solves the first part (the one with class MyCl(myif))
For specific examples doing this, http://wiki.python.org/moin/boost.python/OverridableVirtualFunctions
For the line myif c = MyCl(); you need to expose your python (module) to C++. There are examples here http://wiki.python.org/moin/boost.python/EmbeddingPython
Based upon the (very helpful) answer by Eudoxos I've taken his code and extended it such that there is now an embedded interpreter, with a built-in module.
This answer is the Boost.Python equivalent of my SWIG based answer.
The headerfile myif.h:
class myif {
public:
virtual float myfunc(float a) const { return 0; }
virtual ~myif() {}
};
Is basically as in the question, but with a default implementation of myfunc and a virtual destructor.
For the Python implementation, MyCl.py I have basically the same as the question:
import myif
class MyCl(myif.myif):
def myfunc(self,a):
return a*2.0
This then leaves mymain.cc, most of which is based upon the answer from Eudoxos:
#include <boost/python.hpp>
#include <iostream>
#include "myif.h"
using namespace boost::python;
// This is basically Eudoxos's answer:
struct MyIfWrapper: myif, wrapper<myif>{
float myfunc(float a) const {
if(this->get_override("myfunc"))
return this->get_override("myfunc")(a);
else
return myif::myfunc(a);
}
};
BOOST_PYTHON_MODULE(myif){
class_<MyIfWrapper,boost::noncopyable>("myif")
.def("myfunc",&myif::myfunc)
;
}
// End answer by Eudoxos
int main( int argc, char ** argv ) {
try {
// Tell python that "myif" is a built-in module
PyImport_AppendInittab("myif", initmyif);
// Set up embedded Python interpreter:
Py_Initialize();
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
PySys_SetPath(".");
main_namespace["mycl"] = import("mycl");
// Create the Python object with an eval()
object obj = eval("mycl.MyCl()", main_namespace);
// Find the base C++ type for the Python object (from Eudoxos)
const myif &b=extract<myif>(obj)();
std::cout << b.myfunc(5) << std::endl;
} catch( error_already_set ) {
PyErr_Print();
}
}
The key part that I've added here, above and beyond the "how do I embed Python using Boost.Python?" and "how do I extend Python using Boost.python?" (which was answered by Eudoxos) is the answer to the question "How do I do both at once in the same program?". The solution to this lies with the PyImport_AppendInittab call, which takes the initialisation function that would normally be called when the module is loaded and registers it as a built-in module. Thus when mycl.py says import myif it ends up importing the built-in Boost.Python module.
Take a look at Boost Python, that is the most versatile and powerful tool to bridge between C++ and Python.
http://www.boost.org/doc/libs/1_48_0/libs/python/doc/
There's no real way to interface C++ code directly with Python.
SWIG does handle this, but it builds its own wrapper.
One alternative I prefer over SWIG is ctypes, but to use this you need to create a C wrapper.
For the example:
// myif.h
class myif {
public:
virtual float myfunc(float a);
};
Build a C wrapper like so:
extern "C" __declspec(dllexport) float myif_myfunc(myif* m, float a) {
return m->myfunc(a);
}
Since you are building using C++, the extern "C" allows for C linkage so you can call it easily from your dll, and __declspec(dllexport) allows the function to be called from the dll.
In Python:
from ctypes import *
from os.path import dirname
dlldir = dirname(__file__) # this strips it to the directory only
dlldir.replace( '\\', '\\\\' ) # Replaces \ with \\ in dlldir
lib = cdll.LoadLibrary(dlldir+'\\myif.dll') # Loads from the full path to your module.
# Just an alias for the void pointer for your class
c_myif = c_void_p
# This tells Python how to interpret the return type and arguments
lib.myif_myfunc.argtypes = [ c_myif, c_float ]
lib.myif_myfunc.restype = c_float
class MyCl(myif):
def __init__:
# Assume you wrapped a constructor for myif in C
self.obj = lib.myif_newmyif(None)
def myfunc(a):
return lib.myif_myfunc(self.obj, a)
While SWIG does all this for you, there's little room for you to modify things as you please without getting frustrated at all the changes you have to redo when you regenerate the SWIG wrapper.
One issue with ctypes is that it doesn't handle STL structures, since it's made for C. SWIG does handle this for you, but you may be able to wrap it yourself in the C. It's up to you.
Here's the Python doc for ctypes:
http://docs.python.org/library/ctypes.html
Also, the built dll should be in the same folder as your Python interface (why wouldn't it be?).
I am curious though, why would you want to call Python from inside C++ instead of calling the C++ implementation directly?