So I am writing a Python API for a C++ library using Cython. I have three classes with almost identical functionality: A, B, and C. This difference is only how one of their objects is built on initialization and some constants.
Originally, I have wrote them all out as separate classes and then could define them via extern in my cython code. This compiles and works well but there is a lot of repeated code and I would really like this project to be more DRY.
So I decided to write a base class for A, B, and C that implemented most of the functionality once. However, I needed to template that base class and this is causing me a nightmare when I try to define everything in Cython. Here is a toy example of what I'm talking about (ignore missed semi-colons etc, if you find them). This is my "classes.h" file
int library_method_load(std::string file_name){
return std::string.length();
}
template <class T>
class BaseClass{
public:
T important_obj;
BaseClass(std::string file_name){ important_obj = library_method(file_name);};
virtual T library_method(std::string file_name) = 0;
// Important logicks...
~BaseClass(){};
}
class A : public BaseClass<int> {
A(std::string file_name): BaseClass<int>(file_name){};
int library_method(std::string file_name){ return library_method_load(file_name);};
~A(){};
}
When I try to wrap this, if I don't tell cython about the base class, I get undefined symbols. If I try to define the base class, the templating causes problems. The latter could be due to the fact that I don't know the syntax properly for inheriting templated base classes.
Here is my current attempt
#distutils: language = c++
from libcpp.string cimport string
cdef extern from "classes.h":
cppclass BaseClass[T]:
BaseClass(string file_name)
cdef extern from "classes.h":
cppclass A(BaseClass[int]):
A(string file_name)
cdef class PyBase:
cdef BaseClass* wrapped
cdef class PyA(PyBase):
def __cinit__(self, string file_name):
self.wrapped = <BaseClass[int]*> new A(file_name)
Doing this gives me the following compiler error:
Error compiling Cython file:
------------------------------------------------------------ ...
cdef class PyA(PyBase):
def __cinit__(self, string file_name):
self.wrapped = <BaseClass[int]*> new A(file_name)
^
wrapper.pyx:23:23: Cannot assign type 'BaseClass[int] *' to
'BaseClass[T] *' Traceback (most recent call last): File "setup.py",
line 9, in
setup(name="test", version="1.0.0", ext_modules=cythonize([rk])) File
"/home/jacob/anaconda3/lib/python3.6/site-packages/Cython/Build/Dependencies.py",
line 1027, in cythonize
cythonize_one(*args) File "/home/jacob/anaconda3/lib/python3.6/site-packages/Cython/Build/Dependencies.py",
line 1149, in cythonize_one
raise CompileError(None, pyx_file) Cython.Compiler.Errors.CompileError: wrapper.pyx
Does anyone know how to do this? Clearly my template substituion is off. Should I stick with the repetitive code instead? Are there other clever solutions?
One solution is to drop PyBase because in cdef BaseClass* wrapped the template argument for BaseClass is missing, which makes the line meaningless.
E.g.:
cdef class PyA(PyBase):
cdef A* wrapped
def __cinit__(self, string file_name):
self.wrapped = new A(file_name)
(I am not sure though if you can pass a C++ object into __init__ and __cinit__)
Related
I am new to the Cython/C++ interface and currently studying the examples in the documentation. I am trying to find a way to wrap a C++ struct (instead of C/C++ native types) that is used as the type specifier of the argument of a C++ class constructor. Then I can instantiate a C++ class on the Python side.
First, the rectangle.cpp file which is a simplified version of the example with only constructors. Here I introduce a struct to be used as the type of the argument for the constructor. (I merged the .h header file with .cpp just to make the post shorter. This compiles normally,though.)
rectangle.cpp
#include <iostream>
namespace shapes {
struct Point {
int x0;
int y0;
int x1;
int y1;
};
class Rectangle {
public:
int x0, y0, x1, y1;
Rectangle();
Rectangle(Point point);
~Rectangle();
};
// Default constructor
Rectangle::Rectangle () {}
Rectangle::Rectangle (Point point) {
this->x0 = point.x0;
this->y0 = point.y0;
this->x1 = point.x1;
this->y1 = point.y1;
}
// Destructor
Rectangle::~Rectangle () {}
}
Second, the rectangle.pxd file which is basically a copy from the example, except that I included a cdef struct Point to declare the structure. (Not sure if this is a correct approach.)
rectangle.pxd
# Declare the class with cdef
cdef extern from "rectangle.cpp" namespace "shapes":
cdef struct Point:
int x0
int y0
int x1
int y1
cdef cppclass Rectangle:
Rectangle() except +
Rectangle(Point) except + # Not sure if this is correct
int x0, y0, x1, y1
Third, the rect.pyx file
rect.pyx
# distutils: language = c++
from rectangle cimport Rectangle, Point
# Create a Cython extension type which holds a C++ instance
# as an attribute and create a bunch of forwarding methods
# Python extension type.
cdef class PyRectangle:
cdef Point point # Not sure if this is correct
cdef Rectangle c_rect # Hold a C++ instance which we're wrapping
def __cinit__(self, Point point):
self.c_rect = Rectangle(point)
Finally, the setup.py which is identical to the example.
setup.py
from setuptools import setup
from Cython.Build import cythonize
setup(ext_modules=cythonize("rect.pyx"))
If I compile using the standard command python3 setup.py build_ext --inplace, the following error pops out. (The original example can compile and run without any problem.)
[1/1] Cythonizing rect.pyx
Error compiling Cython file:
------------------------------------------------------------
...
PyTypeObject *Py_TYPE(obj)
bint PyMapping_Check(obj)
object PyErr_Format(exc, const char *format, ...)
#cname("__pyx_convert__from_py_shapes::Point")
cdef Point __pyx_convert__from_py_shapes::Point(obj) except *:
^
------------------------------------------------------------
FromPyStructUtility:11:41: Expected an identifier or literal
Traceback (most recent call last):
File "setup.py", line 5, in <module>
setup(ext_modules=cythonize("rect.pyx"))
File "/usr/lib/python3/dist-packages/Cython/Build/Dependencies.py", line 877, in cythonize
cythonize_one(*args)
File "/usr/lib/python3/dist-packages/Cython/Build/Dependencies.py", line 997, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: rect.pyx
I am sure there are mistakes in wrapping the struct, since the error is triggered by FromPyStructUtility. But I haven't found a straightforward answer/solution to this particular scenario. If I want to keep the C++ code as it is, i.e., insisting on passing a struct to the constructor, how do I implement this on the Cython side? Your comments/suggestions/tutorials will be highly appreciated.
I am fighting with CYTHON, trying to make a copy of a C++ class in a Python class.
I the .pxd I have:
cdef extern from "MyCppClass.h":
cdef cppclass MyCppClass:
MyCppClass ()
int Copy(MyCppClass * Source)
cdef class MyPyClass:
cdef MyCppClass * thisptr
cdef copy(self, MyCppClass *s)
and in the .pyx I have
cdef class MyPyClass:
def __init__(self):
self.thisptr = new MyCppClass ()
cdef copy(self, MyCppClass * s):
self.thisptr.Copy(s)
The compilation is ok.
Then I try to use it in another Cython module:
A=new MyCppClass()
pA=MyPyClass()
… do some stuff on A
p1.copy(A)
And I get the usual “cannot convert to Python object”
I have tried a lot of things to copy, like :
cdef getThisptr(self):
return self.thisptr
A=new MyCppClass()
pA=MyPyClass()
B=pA.getThisptr()
B.Copy(A)
But nothing works!
It sounds very basic in Cython, but I am still waiting for a solution…
You're missing cimport. In order for Cython to know about your cdef defined types and functions you need to add
from module_name cimport MyPyClass
Otherwise it just assumes that Copy is a Python function accepting a Python object.
The relevant bit of documentation
I use Cython 0.19.2 (and Python 2.7.1) to expose C++ classes to Python.
As a first try, i did a test with the 'Rectangle' class example of the documentation.
http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html
I have a crash which I don't understand.
I have tried to simplify the code the the max. but I still have the problem.
Here is my pyx file, the C++ sources are just a cut&paste from the python/cython's documentation.
# distutils: language = c++
# distutils: sources = Rectangle.cpp
cdef extern from "Rectangle.h" namespace "shapes":
cdef cppclass Rectangle:
pass
cdef class PyRectangle:
cdef Rectangle* thisptr
I just want to declare a class with a thisptr, which points the the C++ Rectangle class instance.
When i try to compile the program with:
cython -a --cplus rect.pyx
I have the following crash:
Error compiling Cython file:
------------------------------------------------------------
...
cdef extern from "Rectangle.h" namespace "shapes":
cdef cppclass Rectangle:
pass
cdef class PyRectangle:
cdef Rectangle* thisptr
^
------------------------------------------------------------
rect.pyx:9:7: Compiler crash in AnalyseDeclarationsTransform
File 'ModuleNode.py', line 101, in analyse_declarations: ModuleNode(rect.pyx:1:0,
full_module_name = 'rect')
File 'Nodes.py', line 382, in analyse_declarations: StatListNode(rect.pyx:4:0)
File 'Nodes.py', line 4251, in analyse_declarations: CClassDefNode(rect.pyx:8:5,
as_name = u'PyRectangle',
class_name = u'PyRectangle',
module_name = u'',
visibility = u'private')
File 'Nodes.py', line 382, in analyse_declarations: StatListNode(rect.pyx:9:7)
File 'Nodes.py', line 1208, in analyse_declarations: CVarDefNode(rect.pyx:9:7,
modifiers = [...]/0,
visibility = u'private')
Compiler crash traceback from this point on:
File "/home/xxx/local/python2.7.1/site-packages/Cython/Compiler/Nodes.py", line 1208, in analyse_declarations
self.entry.doc = embed_position(self.pos, self.doc)
AttributeError: 'CVarDefNode' object has no attribute 'doc'
I have tried to compile with pyrex, setup.py, ... everything. But I still have the same error.
Is there something I'm missing?
Thanks
Alright, I finally fixed it. My version of Python 2.7.1 simply doesn't work with the last Cython 0.19.2. I upgraded to 2.7.6 and it works.
This seems like it should be a very simple question, and I've seen similar discussions here, but nothing that quite tackles this problem. I have a class written in c++ that I would like to access with cython. The simple example below illustrates the problem, it compiles just fine, however, I get an ImportError when I use it.
//element.h
template <typename T>
class element{
public:
element(T);
~element();
T data;
};
and
//element.cc
#include "element.h"
template <typename T>
element<T>::element(T _data){
data = _data;
}
template <typename T>
element<T>::~element(){
}
it's accessed through the following simple cython
cdef extern from "element.h":
cdef cppclass element[T]:
element(T) except +
T data
cdef element[int] *el = new element[int](3)
print el.data
and compiled in place with
from distutils.core import setup, Extension
from Cython.Distutils import build_ext
ext_modules = [Extension("example",
['example.pyx','./element.cc'],
language = "c++")]
setup(cmdclass = {'build_ext':build_ext},
name = 'example',
ext_modules = ext_modules)
However, when I try to import the resulting shared library, I get
ImportError: .../example.so: undefined symbol: _ZN7elementIiEC1Ei
If I simply strip out all the templating and force ints for example, the code compiles just fine (as before) but this time it runs. So, in other words, this works fine.
//element.h
class element{
public:
element(int);
~element();
int data;
};
and
//element.cc
#include "element.h"
element::element(int _data){
data = _data;
}
element::~element(){
}
and
//example.pyx
cdef extern from "element.h":
cdef cppclass element:
element(int) except +
int data
cdef element *el = new element(3)
print el.data
What am I doing wrong with the templating in the first case?
You need to implement template in one header file instead of split into element.cc. When I run c++file command it shows that compiler failed to link element constructor definition
$ c++filt _ZN7elementIiEC1Ei
element<int>::element(int)
I am currently using SWIG to make the implementation of small modules easier in my main C++ programm. The class architecture is as follow :
foo.hpp :
class Foo
{
public:
virtual int pureFunc() = 0;
int func() { return (42); }
};
file.i :
%module(directors="1") foo
%feature("director") Foo;
%{
#include "foo.hpp"
%}
%include "foo.hpp"
file.py :
import sys
sys.path.append('.')
import foo
class Bar(foo.Foo):
def __init__(self):
pass
def pureFunc(self):
return 21
lol = Bar()
print lol.pureFunc()
print lol.func()
I then generate the swig wrappers using the following command :
swig -python -c++ file.i
and compile the .so like this :
g++ -fPIC -shared -o _foo.so file_wrap.cxx -I/usr/include/python2.7 -lpython2.7
And when I try to run the python script I get the following error :
# python file.py
21
Traceback (most recent call last):
File "file.py", line 13, in <module>
print lol.func()
File "/home/volent/dev/CPP/cython/foo.py", line 84, in func
def func(self): return _foo.Foo_func(self)
TypeError: in method 'Foo_func', argument 1 of type 'Foo *'
# _
That shows that the use of the pure method is working but I can't use the one already defined in the .hpp file.
I have tried to read the SWIG documentation and the only things I see about abstract class and inheritance are 34.4.7 C++ Classes and 34.4.8 C++ Inheritance. And I can't see anything mentionning a case like that.
You have forgotten to call the __init__ method of the parent class of Bar. Replace your __init__ method with this:
class Bar(foo.Foo):
def __init__(self):
super(Bar,self).__init__()
This should let your Bar class know about the Foo method.