I have a C++ class. It's made up of one .ccp file and one .h file. It compiles (I can write a main method that uses it successfully in c++). How do I wrap this class with Cython to make it available in Python?
I've read the docs and don't follow. They talk about generating the cpp file. When I've tried to follow the docs, my already existing cpp gets blown away...
What am I meant to put in the pyx file? I've been told the class definition but how much of it? Just the public methods?
Do I need a .pxd file? I don't understand when this file is or isn't required.
I've tried asking these question in the #python IRC channel and can't get an answer.
Even Cython is generally for use with C, it can generate C++ code, too. When compiling, you add the --cplus flag.
Now, creating a wrapper for the class is simple and not much different from wrapping a structure. It mainly differs from declaring the extern, but that's not much difference at all.
Suppose you have a class MyCppClass in mycppclass.h.
cdef extern from "mycppclass.h":
cppclass MyCppClass:
int some_var
MyCppClass(int, char*)
void doStuff(void*)
char* getStuff(int)
cdef class MyClass:
# the public-modifier will make the attribute public for cython,
# not for python. Maybe you need to access the internal C++ object from
# outside of the class. If not, you better declare it as private by just
# leaving out the `private` modifier.
# ---- EDIT ------
# Sorry, this statement is wrong. The `private` modifier would make it available to Python,
# so the following line would cause an error es the Pointer to MyCppClass
# couldn't be converted to a Python object.
#>> cdef public MyCppClass* cobj
# correct is:
cdef MyCppClass* obj
def __init__(self, int some_var, char* some_string):
self.cobj = new MyCppClass(some_var, some_string)
if self.cobj == NULL:
raise MemoryError('Not enough memory.')
def __del__(self):
del self.cobj
property some_var:
def __get__(self):
return self.cobj.some_var
def __set__(self, int var):
self.cobj.some_var = var
Note that the new keyword is only available when the --cplus flag is set, otherwise use malloc from <stdlib.h> by externing it.
Also note that you don't need to dereference the pointer (->) to call the method. Cython tracks the object's type and applies what fits.
.pxd files are for seperating declarations from implementation, or to avoid namespace colliding. Imagine you'd like to name you Python-wrapper like the C++ class. Simply put in your .pxd file the extern declarations and cimport the pxd file in the .pyx.
cimport my_pxd
cdef my_pxd.SomeExternedType obj
Note that you can not write implementations in a .pxd file.
So after lots of poking, trial and error, screaming and tearing my hair out, I finally got this to work. First though, I had to re-write my C++ into C, which for me really just involved converting all my std::string variables to char* and keeping track of some lengths.
Once done I had my .h and .c files. I wanted to make a single function from the C code available in Python. It turns out that Cython can compile your C files into the extension for you and link any libraries all in one go, so starting with my setup.py, it ended up looking like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules=[
Extension("myext",
["myext.pyx", "../stuff.c"],
libraries=["ssl", "crypto"]
)
]
setup(
name = "myext",
cmdclass = {"build_ext": build_ext},
ext_modules = ext_modules
)
As you can see, the second argument to the Extension simply lists all the files that need to be compiled, Cython works out how to compile them depending on their file extension as far as I can tell. The libraries array tells the Cython compiler what needs to be linked in (in this case I was wrapping some crypto stuff that I couldn't seem to mimick directly through existing Python libs).
To actually make my C function available in the .pyx file, you write a small wrapper in the .pxd. My myext.pxd looked as below:
cdef extern from "../stuff.h":
char* myfunc(char* arg1, char* arg2, char* arg3)
In the .pyx you then use the cimport declaration to import this function, which is then available for use as if it were any other Python function:
cimport myext
def my_python_func(arg1, arg2, arg3):
result = myext.myfunc(arg1, arg2, arg3)
return result
When you build this (on Mac at least) you get a .so that you can import in python and run the functions from the .pyx. There may be better, more correct way to get this all working but that comes from experience and this was a first encounter that I managed to work out. I'd be very interested on pointers where I may have gone wrong.
Update:
After further use of Cython, I found it was super simple to integrate it with C++ too, once you know what you're doing. Making C++'s string available is as simple as from libcpp.string cimport string in your pyx/pyd. Declaring the C++ class is similarly easy as:
cdef extern from "MyCPPClass.h":
cdef cppclass MyCPPClass:
int foo;
string bar;
Sure you have to basically redeclare the .h definition of your class in a Pythonic format, but that's a small price to pay for getting access to your already written C++ functions.
Cython is mainly for C development, to integrate C++ with Python I would recommend Boost.Python. Their excellent documentation should get you started pretty quickly.
The answers above have more or less answered the OP's question.
But its now passing the mid of 2020 and I'd thought to put a contribution (in the form of resource summary) here for those who want to explore Python-C++ bindings via python packages that can be 'pip installed' (or in the PyPI).
I'd reckon for calling C++ from Python / wrapping over third party libs , one can look at cppyy. Its a relatively young project but it certainly looks promising and backed with a modern compiler. See the link below:
https://cppyy.readthedocs.io/en/latest/
Also, there's always pybind11 ....
https://github.com/pybind/pybind11
Cython also supports C++ natively (for most of the C++ languages) ; for more information see https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html
Related
From the docs :
cppyy is an automatic, run-time, Python-C++ bindings generator, for
calling C++ from Python and Python from C++.
(Emphasis mine)
I don't see any instructions for doing the same, however, so is it possible to call Python via C++ using cppyy?
As a qualifier, since I don't know from where you obtained cppyy, the main code that was at the time the reason for typing that sentence does not exist in cppyy master, but does exist in its historic home of PyROOT. This so-called "class generator" plugin allows Cling to "see" Python classes as C++ classes, for straightforward callbacks and even inheritance of C++ classes from Python ones. See this publication (page 3) for some examples: https://www.researchgate.net/publication/288021806_Python_in_the_Cling_World
This code was not ported over to cppyy standalone b/c the class generator relies on interactive use (specifically, dynamic scopes), so only works from Cling, not compiled code, and there is no way (yet) to drop into the Cling prompt from Python (vice versa works).
The reason why that sentence is still there even without the class generator, is that cppyy has since grown a multitude of other ways to call Python from C++ (these have been backported into PyROOT). Examples include C-style function pointers, C++ std::function<> objects, lambdas, and cross-language inheritance. Moreover, these can all be used by importing cppyy into embedded Python (and thus be used from compiled C++).
See e.g. these examples in the documentation: callbacks and cross-inheritance. Cross-inheritance is probably the easiest to use: just define an abstract interface, implement it in Python, pass the pointer to C++ and use it like you would with any pointer-to-interface in C++. For callbacks, declare an extern function pointer (or std::function object) in a header, pass that in a cppyy.include in embedded Python, assign a Python function to that pointer, then call it in C++ as desired.
Callbacks can be made quite sophisticated, assuming that the C++ side can handle it. For example, by providing annotations on the Python side, Python functions can instantiate C++ function pointer templates. Take the completely generic callback in C++ below, which accepts any arguments and producing any result:
>>> import cppyy
>>> cppyy.cppdef("""\
... template<typename R, typename... U, typename... A>
... R callT(R(*f)(U...), A&&... a) {
... return f(a...);
... }""")
True
>>> def f(a: 'int') -> 'double':
... return 3.1415*a
...
>>> cppyy.gbl.callT(f, 2)
6.283
>>> def f(a: 'int', b: 'int') -> 'int':
... return 3*a*b
...
>>> cppyy.gbl.callT(f, 6, 7)
126
>>>
The final way of calling from C++ into Python is indeed not documented b/c it is (still) only available for CPython/cppyy, not PyPy/_cppyy and the naming is implementation-specific as well: CPyCppyy/API.h.
This header is meant to be included in C++ code, allowing the boxing and unboxing of cppyy-bound instances from C++, custom converters and executors, memory management, and parameter packing for stub functions. There are also a couple of convenience functions for dealing with one-offs. For example:
import cppyy
def pyfunc():
return 42
cppyy.cppdef("""\
#include "CPyCppyy/API.h"
int cppfunc() {
return (int)CPyCppyy::Eval("pyfunc()");
}""")
print(cppyy.gbl.cppfunc())
(although the example here is run from Python for convenience, this can all be called from embedded Python in compiled C++ as well).
I do not know if it answer to your question (from your title I understand that you want to call python function from c++ side, but later it seems you ask specifically about cpppy) but you can do the binding with pybind11. This package is heavily used and is being used in a lot of case as an alternative to swig (to understand the differences have a look at this open thread in the TensorFlow community). These are the two most used packages for Python-C++ binding.
To see how to call a python function in C++ for Pybind11 have a look at this question, while to see how to call a python function in C++ for swig have a look at this question.
I'm trying to wrap a cpp application and the source code contains the following in a header file
using Time = cppClassDefinition<withT>
...
void setDefaultTime(Time x)
What would be the Cython equivalent for this?
I've tried
cdef extern from "headerfile.h" namespace "ns":
ctypedef cppClassDefinition<withT> Time
to no success. Although Cython does not complain at this step, it throws a compilation error when I try to use the function setDefaultTime(1.0). The error states "Cannot assign type 'double' to 'Time'. In the CPP code however, this seems to be working fine.
I've also tried
cdef extern from "headerfile.h" namespace "ns":
cdef cppclass Time:
pass
and that also failed. Any suggestions? Is this possible using Cython?
As you suggest in the question, you should be able to use use
ctypedef cppClassDefinition[withT] Time
since using ... = ... in this context is equivalent to a typedef. (Note the change to square brackets compared to the code in your question).
I believe the problem is instead with your attempt to do setDefaultTime(1.0). Cython has no way of knowing that double can be converted to Time and there isn't any way of telling it about implicit C++ conversions.
The easiest way round this is to just tell Cython that the function signature is
void setDefaultTime(double x)
(You can leave the C++ signature as is). This will satisfy Cython, and then the C++ code it generates should end up working correctly provided that double can be implicitly converted to Time (as the question implies)
I need to write Python2 wrapper for proprietary library, consists of few .h files (I made one big), bunch of .dll and one .lib file to link all this stuff.
I think I need API level, because of all this `typedef' in .h files
Script to create wrapper: build_wrapper.py
from cffi import FFI
import setuptools
ffibuilder = FFI()
ffibuilder.set_unicode(enabled_flag=True)
with open(os.path.join(curdir, 'include', 'ScadWrapper.h'), 'r') as f:
source = f.read()
ffibuilder.set_source('_wrapper', source,
extra_link_args=[r'C:\Documents\python\pyScadApi\pyScadApi\include\SCADAPIX.lib', ],
source_extension='.cpp')
if __name__ == '__main__':
ffibuilder.compile(verbose=True)
This runs without errors Creating library .\Release\_wrapper.lib and object .\Release\_wrapper.exp
But, for example,
from _wrapper import ffi, lib
lp_api = ffi.new('ScadAPI *')
r = lib.ApiCreate(lp_api)
Fails with
lp_api = ffi.new('ScadAPI *')
ffi.error: undefined type name
ScadAPI *
ScadAPI defined as
struct APIHandle_tag;
typedef APIHandle_tag * ScadAPI;
in ScadWrapper.h
You are never calling ffibuilder.cdef(). That's why the lib object is empty: it doesn't know about any type or function.
Sorry about being brief. My point is that the basics are explained on http://cffi.readthedocs.io/en/latest/overview.html. Following the "real example", the idea is to write in the cdef() only the pieces that are interesting to you, one function after the other, and the type declarations with suitable usage of ...;. Every function or type you write in cdef() becomes available for call (via lib.the_function_name()) or for ffi operations (ffi.new(), etc.).
There are different approaches to cffi (not directly documented or supported) that try to expose a whole large library at once, without needing any function-by-function work, however small that is. The idea is to extract from a .h file (possibly preprocessed by gcc -E) something that can be accepted by the cdef(). The drawback, particularly if you use gcc -E, is that the result is likely to work only on your exact OS. Moreover, such an approach appears faster but often isn't: it seems to avoid a per-function work, but that's never true, because if you are going to use that function somewhere, then you'll take time to write that function call anyway.
I have a task to interface with a dll from a third party company using C++.
The dll package comes with:
the dll itself
a sample Java implementation - consists of a java wrapper(library) generated using the SWIG tool and the the java source file
documentation that states all the public datatypes, enumeration and member functions.
My other colleague is using Java(based on the example in package) to interface with the dll while I'm asked to use C++. The Java example looks straight forward... just import the wrapper and instantiate any class described in the docs..
More info on the dll:
From the docs, it says the dll was programmed using C++
From a hexdump, it shows that it was compiled using VC90 (VS C++ 2008 right?) and something from Dinkumware.
From a depends.exe output, the functions seems to be wrapped under JNI. For example: _Java_mas_com_oa_rollings_as_apiJNI_Server_1disconnect#20
My dilemma:
The dll company is not changing anything in the dll and not providing any other info.
How do i use the member functions in the class from the dll?
I did some simple LoadLibrary() and GetProcAddress and manage to get the address of the public member functions.
But i dunno how to use the functions that has the datatype parameters defined in the dll. For example:
From the docs, the member function is defined as:
void Server::connect(const StringArray, const KeyValueMap) throw(std::invalid_argument,std::out_of_range)
typedef std::map Server::KeyValueMap
typedef std::vector Server::StringArray
how do i call that function in C++. The std::map and std::vector in my compiler (VS 2005) has different functions listing that the one in the dll. For example, from the depends.exe output:
std::map // KeyValueMap - del, empty, get, has_1key,set
std::vector // StringArray - add, capacity, clear, get, isEMPTY, reserve, set, size
Any advice/strategy on how i should solve this? Is it possible to simply instantiate the class like the Java example?
If you are trying to use VS 2005 to try and interface with a DLL that is built using VS2008, your attempts will be mostly doomed unless you can use a plain C interface. Given your description, this is not the case; The runtime libraries differ between VS2005 and VS2008 so there is little chance that the object layout has stayed the same between compilers. The 'something from Dinkumware' that you're referring to is most likely the C++ standard library as ISTR that Microsoft uses the Dinkumware one.
With your above example you're also missing several important pieces of information - the types you describe (Server::StringArray and Server::KeyValueMap) are standard library containers. OK fine, but standard library containers of what? These containers are templates and unless you know the exact types these templates have been instantiated with, you're a little stuck.
Is this DLL intended to be called from C++ at all? The fact that it export a JNI interface suggests that it might not be in the first place. Does it export any other public symbols apart from those that are of the format _Java_...?
Of course if there is no other way in and you must use C++ instead of Java, you might want to look into embedding a JVM into your C++ app and use that to call through to the C++ dll. It's not what I'd call an elegant solution but it might well work.
I don't quite understand the use of C++ standard library data types here. How can Java code provide a std::map argument? Are the arguments you pass in always just "opaque" values you would get as output from a previous call to the library? That's the only way you're going to be able to make it work from code under a different runtime.
Anyway...
When you make a JNI module, you run javah.exe and it generates a header file with declarations like:
JNIEXPORT void JNICALL Java_Native_HelloWorld(JNIEnv *, jobject);
Do you have any such header file for the module?
These symbols are exported as extern "C" if I recall correctly, so if you can get the correct signatures, you should have no issues with name mangling or incompatible memory allocators, etc..
The "#20" at the end of the method signature means that the function is declared "stdcall" and that 20 bytes are put on the stack when the function is called. All these methods should start with a JNIEnv* and a jobject, these will total 8 bytes I believe, on a 32-bit environment, so that leaves 12 bytes of parameters you will need to know in order to generate a correct function prototype.
Once you figure out what the parameters are, you can generate something like this:
typedef void (__stdcall *X)(JNIEnv *, jobject, jint i, jboolean b);
Then, you can cast the result of GetProcAddress to an X and call it from your C++ code.
X x = (X)GetProcAddress(module, "name");
if (x) x(params...);
Unfortunately, what you have doesn't quite look like what I have seen in the past. I am used to having to deal with Java data types from C/C++ code, but it looks like this module is dealing with C++ data types in Java code, so I don't know how relevant any of my experience is. Hopefully this is some help, at least.
I have a program in C++ that uses the cryptopp library to decrypt/encrypt messages.
It offers two interface methods encrypt & decrypt that receive a string and operate on it through cryptopp methods.
Is there some way to use both methods in Python without manually wrapping all the cryptopp & files included?
Example:
import cppEncryptDecrypt
string foo="testing"
result = encrypt(foo)
print "Encrypted string:",result
If you can make a DLL from that C++ code, exposing those two methods (ideally as "extern C", that makes all interfacing tasks so much simpler), ctypes can be the answer, not requiring any third party tool or extension. Otherwise, it's your choice between cython, good old SWIG, SIP, Boost, ... -- many, many such 3rd party tools will let your Python code call those two C++ entry points without any need for wrapping anything else but them.
As Alex suggested you can make a dll, export the function you want to access from python and use ctypes(http://docs.python.org/library/ctypes.html) module to access e.g.
>>> libc = cdll.LoadLibrary("libc.so.6")
>>> printf = libc.printf
>>> printf("Hello, %s\n", "World!")
Hello, World
or there is alternate simpler approach, which many people do not consider but is equally useful in many cases i.e. directly call the program from command line. You said you have already working program, so I assume it does both encrypt/decrypt from commandline? if yes why don't you just call the program from os.system, or subprocess module, instead of delving into code and changing it and maintaining it.
I would say go the second way unless it can't fulfill your requirements.