I am using SWIG to access C++ code from Python. How do I elegantly wrap a function that returns values in variables passed by reference like
void set(double&a) {
a = 42.;
}
I could not find out how to do this. In the best case I'd be able to use the function in Python with Python floats:
>>> b = 2.
>>> set(b)
>>> print b
42.0
At the moment it gives me a TypeError: in method 'TestDouble_set', argument 2 of type 'double &'.
Do it this way:
Your swig interface file:
%include <typemaps.i>
%apply double& INOUT { double& a };
void set(double& a);
Usage in python script:
a = 0.0
a = set(a)
print a
If your function returns something (instead of being a void), do the below in python
ret, a = set(a)
Checkout the documentation for typemaps in swig. You can do INPUT, OUTPUT & INOUT for arguments. HTH
Note that this solution relies on the SWIG-provided OUTPUT typemaps defined in typemaps.i library, which pre-defines the typemaps being used by the %apply command above.
typemaps.i defines input/output typemaps for C++ primitive types (see the above link to the SWIG documentation for more info); however, you have into include the typemaps.i library in your interface file for SWIG to use them. (Hence why some commenters likely found the original solution wasn't working for them.)
Note that the accepted (correct) solution relies on the SWIG-provided OUTPUT typemaps defined in typemaps.i library, which pre-defines the typemaps being used by the %apply command above.
typemaps.i defines input/output typemaps for C++ primitive types (see the above link to the SWIG documentation for more info); however, you have to include the typemaps.i library in your interface file for SWIG to use them. (Hence why some commenters likely found the original solution wasn't working for them.)
hmm - are you using the latest version of SWIG? The documentation seems to indicate that this works -- from the manual:
C++ references are supported, but SWIG will treat them as pointers. For example, a declaration like this :
class Foo {
public:
double bar(double &a);
}
will be accessed using a function like this :
double Foo_bar(Foo *obj, double *a) {
obj->bar(*a);
}
Functions returning a reference will be mapped into functions returning pointers.
I don't know how you map that on the python side...does python have something like perl references?
It has been very helpful. I recently encountered similar issue, I found this on swig site: (the Return by value part)
I applied it to my interface file (c++ method returning double* -convert it to c#)
it seems working.
Related
From the docs :
cppyy is an automatic, run-time, Python-C++ bindings generator, for
calling C++ from Python and Python from C++.
(Emphasis mine)
I don't see any instructions for doing the same, however, so is it possible to call Python via C++ using cppyy?
As a qualifier, since I don't know from where you obtained cppyy, the main code that was at the time the reason for typing that sentence does not exist in cppyy master, but does exist in its historic home of PyROOT. This so-called "class generator" plugin allows Cling to "see" Python classes as C++ classes, for straightforward callbacks and even inheritance of C++ classes from Python ones. See this publication (page 3) for some examples: https://www.researchgate.net/publication/288021806_Python_in_the_Cling_World
This code was not ported over to cppyy standalone b/c the class generator relies on interactive use (specifically, dynamic scopes), so only works from Cling, not compiled code, and there is no way (yet) to drop into the Cling prompt from Python (vice versa works).
The reason why that sentence is still there even without the class generator, is that cppyy has since grown a multitude of other ways to call Python from C++ (these have been backported into PyROOT). Examples include C-style function pointers, C++ std::function<> objects, lambdas, and cross-language inheritance. Moreover, these can all be used by importing cppyy into embedded Python (and thus be used from compiled C++).
See e.g. these examples in the documentation: callbacks and cross-inheritance. Cross-inheritance is probably the easiest to use: just define an abstract interface, implement it in Python, pass the pointer to C++ and use it like you would with any pointer-to-interface in C++. For callbacks, declare an extern function pointer (or std::function object) in a header, pass that in a cppyy.include in embedded Python, assign a Python function to that pointer, then call it in C++ as desired.
Callbacks can be made quite sophisticated, assuming that the C++ side can handle it. For example, by providing annotations on the Python side, Python functions can instantiate C++ function pointer templates. Take the completely generic callback in C++ below, which accepts any arguments and producing any result:
>>> import cppyy
>>> cppyy.cppdef("""\
... template<typename R, typename... U, typename... A>
... R callT(R(*f)(U...), A&&... a) {
... return f(a...);
... }""")
True
>>> def f(a: 'int') -> 'double':
... return 3.1415*a
...
>>> cppyy.gbl.callT(f, 2)
6.283
>>> def f(a: 'int', b: 'int') -> 'int':
... return 3*a*b
...
>>> cppyy.gbl.callT(f, 6, 7)
126
>>>
The final way of calling from C++ into Python is indeed not documented b/c it is (still) only available for CPython/cppyy, not PyPy/_cppyy and the naming is implementation-specific as well: CPyCppyy/API.h.
This header is meant to be included in C++ code, allowing the boxing and unboxing of cppyy-bound instances from C++, custom converters and executors, memory management, and parameter packing for stub functions. There are also a couple of convenience functions for dealing with one-offs. For example:
import cppyy
def pyfunc():
return 42
cppyy.cppdef("""\
#include "CPyCppyy/API.h"
int cppfunc() {
return (int)CPyCppyy::Eval("pyfunc()");
}""")
print(cppyy.gbl.cppfunc())
(although the example here is run from Python for convenience, this can all be called from embedded Python in compiled C++ as well).
I do not know if it answer to your question (from your title I understand that you want to call python function from c++ side, but later it seems you ask specifically about cpppy) but you can do the binding with pybind11. This package is heavily used and is being used in a lot of case as an alternative to swig (to understand the differences have a look at this open thread in the TensorFlow community). These are the two most used packages for Python-C++ binding.
To see how to call a python function in C++ for Pybind11 have a look at this question, while to see how to call a python function in C++ for swig have a look at this question.
I've started to play around with SWIG, so that I can use C libraries in Python. I have this piece of code where I'm passing a Python String into a C function that expects a "void *"
example.h:
char test(void *buf);
example.c:
char test(void *buf) {
char *c = (char*)buf;
return *c;
}
Python:
import example
buf = "hello"
example.test(buf)
When I run it, I get the following error:
TypeError: in method 'test', argument 1 of type 'void *'
However, when I change the "void*" parameter to "char*", it seems to work. I'm a little confused as I thought "void*" matches any kind of pointer. Anyways, I dug around and came across the ctypes library and casted it to a c_void_p (Python: converting strings for use with ctypes.c_void_p()). That didn't seem to work for me.
As a work around I made a wrapper in my swig file:
/* File: example.i */
%module example
%include typemaps.i
%{
#include "example.h"
%}
%include "example.h"
%inline %{
char test_wrapper(char *buf) {
void *voidBuf = (void*)buf;
return test(voidBuf);
}
%}
This seems to work. However, I was wondering if someone could provide an explanation as to why the ctypes approach didn't work. If I was completely off the mark with the ctypes approach, is there a more appropriate way than creating an inline wrapper?
Thanks guys!
Your inline wrapper approach is a sound generalized way to work around functions that aren't quite usable directly. Generally SWIG tries to mimic the behavior of C or C++ inside the target language. In this case though as you observed it doesn't quite do that. I think the reason for this is twofold: firstly it's not obvious how you'd go about doing that in some of the languages SWIG supports (e.g. Java). Secondly, even though you could do something in Python that would work for this specific case in general void* is pretty ambiguous to a C programmer without further guidance about how it is intended to be interpreted/used. There's a mis-match between C and Python here too - strings are reference counted and immutable in Python so it would be very easy to stray into circumstances where the Python behavior breaks in ways that are totally non-obvious to Python programmers.
Using ctypes to perform the cast isn't really a viable solution, SWIG and ctypes have fundamentally very different approaches to wrapping things and typing of arguments. (If you're curious I just wrote a fairly detailed answer looking at interpoerability between them)
The other option in the general case is to user %include <cpointer.i> to generate some code for casting/converting for you, e.g.:
%pointer_cast(type1, type2, name)
However in this instance since you're using char* so the note in the first note documentation applies:
Note: None of these macros can be used to safely work with strings
(char * or char **).
Note: When working with simple pointers, typemaps can often be used to
provide more seamless operation.
And in general I'd favor the point made by the second note, which is that by doing a little more work (e.g. %inline to supply an overload) gives a more intuitive behavior. So in your specific case the only change I'd recommend would be using %rename(test) test_wrapper; to make it into an overload inside Python. (If it were C++ instead of C that you're targeting then you could do it as an overload inside C++)
I want to know if there is any way to expose a C++ class to Python but without building an intermediate shared library.
Here is my desirable scenario. For example I have following C++ class:
class toto
{
public:
toto(int iValue1_, int iValue2_): iValue1(iValue1_), iValue2(iValue2_) {}
int Addition(void) const {if (!this) return 0; return iValue1 + iValue2;}
private:
int iValue1;
int iValue2;
};
I would like to convert somehow this class (or its intance) to a PyObject* in order to send it as paremter (args) to for example PyObject_CallObject:
PyObject* PyObject_CallObject(PyObject* wrapperFunction, PyObject* args)
In the other hand in my python side, I'll have a wrapperFunction which gets the pointer on my C++ class (or its instance) as parameter and it calls its methods or uses its properties:
def wrapper_function(cPlusPlusClass):
instance = cPlusPlusClass(4, 5)
result = instance.Addition()
As you can see, I don't really need/want to have a separate shared library or build a module by boost python. All that I need is to find a way to convert a C++ code to PyObject and send it to python. I cannot find a way to do that by C python libraries, boost or SWIG.
As far as I know, there is no easy way to accomplish this.
To extend Python with C++ with neither a module nor an intermediate library, it would require dynamically loading a library, then importing the functions. This approach is used by the ctypes module. To accomplish the same with C++, one would need to write a ctypes-like library that understood the C++ ABI for the target compiler(s).
To extend Python without introducing a module, an intermediate library could be created that provided a C API that wraps the C++ library. This intermediate library could then be used in Python through ctypes. While it does not provide the exact calling syntax and does introduce an intermediate library, it would likely be less effort than building a ctypes-like library that could interface directly with C++.
However, if an intermediate library is going to be introduced, it may be worthwhile to use Boost.Python, SWIG, or some other C++/Python language binding tool. While many of these tools will introduce the extension via a module, they often provide cleaner calling conventions, better error checking in the binding process, and may be easier to maintain.
I found my answer. Actually what I was searching was pretty similar to this answer (thanks moooeeeep for his comment):
Exposing a C++ class instance to a python embedded interpreter
Following C++ class (Attention! default constructor is mandatory):
class TwoValues
{
public:
TwoValues(void): iValue1(0), iValue2(0) {}
TwoValues(int iValue1, int iValue2): iValue1(iValue1_), iValue2(iValue2_) {}
int Addition(void) const {if (!this) return 0; return iValue1 + iValue2;}
public:
int iValue1;
int iValue2;
};
could be exposed by boost by following macro:
BOOST_PYTHON_MODULE(ModuleTestBoost)
{
class_<TwoValues>("TwoValues")
.def("Addition", &TWOVALUES::Addition)
.add_property("Value1", &TWOVALUES::iValue1)
.add_property("Value2", &TWOVALUES::iValue2);
};
In the other hand I have a python function defined in python_script.py which takes an instance of this class and do something. For example:
def wrapper_function(instance):
result = instance.Addition()
myfile = open(r"C:\...\testboostexample.txt", "w")
output = 'First variable is {0}, second variable is {1} and finally the addition is {2}'.format(instance.Value1, instance.Value2, result)
myfile .write(output)
myfile .close()
Then in C++ side, I can call this function by sending at the same time the instance of my class, like this:
Py_Initialize();
try
{
TwoValues instance(5, 10);
initModuleTestBoost();
object python_script = import("python_script");
object wrapper_function = python_script.attr("wrapper_function");
wrapper_function(&instance);
}
catch (error_already_set)
{
PyErr_Print();
}
Py_Finalize();
Advantages:
I don't need build any shared library or binary
As I'm using Boost, I don't need to be worry about memory management
& reference counting
I don't use shared boost pointer (boost::shared_ptr) to point to the
instance of my class
I have a C++ program:
class X
{
private:
int a;
public:
int func(char *str);
};
Now I want to call the function func in my Python program. How can I do it?
I have successfully called C functions from my Python program using ctypes.
However I can't seem to figure out how to call functions defined inside C++ objects.
I have already worked with ctypes, so I would like to figure out how to accomplishing it that way. However I'm open to other techniques. Also my project puts a constarint that I am not supposed to use wrapper function in my C++ program.
Check out Boost::Python.
You tagged the question ctypes, but you cannot instantiate a C++ object directly from ctypes. You can wrap the C++ object in a C interface and then use ctypes. Or you could write a Python extension module which could include C++ code.
An alternative to Boost.Python is SWIG.
try http://swig.org/
years ago i wrapped some c/c++ code to use them in wxPython with swig.
i cannot remember too much details but i think swig is easy to use.
Hope that helps.
I've got a library written in C++ which I wrap using SWIG and use in python. Generally there is one class with few methods. The problem is that calling these methods may be time consuming - they may hang my application (GIL is not released when calling these methods). So my question is:
What is the simplest way to release GIL for these method calls?
(I understand that if I used a C library I could wrap this with some additional C code, but here I use C++ and classes)
Not having any idea what SWIG is I'll attempt an answer anyway :)
Use something like this to release/acquire the GIL:
class GILReleaser {
GILReleaser() : save(PyEval_SaveThread()) {}
~GILReleaser() {
PyEval_RestoreThread(save);
}
PyThreadState* save;
};
And in the code-block of your choosing, utilize RAII to release/acquire GIL:
{
GILReleaser releaser;
// ... Do stuff ...
}
The real problem is that SWIG is not documented well (I saw hints to use changelog for searching ;) ).
Ok, I found out that I can do inline functions in SWIG and use macros to release/acquire GIL, it looks like this:
%inline %{
void wrappedFunction(OriginalObject *o, <parameters>) {
Py_BEGIN_ALLOW_THREADS
o->originalFunction(<parameters>);
Py_END_ALLOW_THREADS
}
%}
This function is not present in original C++, but available in python module. This is (almost) exactly what I wanted. (what I would like is to wrap original method like python decorator does)
You can use the same API call as for C. No difference. Include "python.h" and call the appoproate function.
Also, see if SWIG doesn't have a typemap or something to indicate that the GIL shuold not be held for a specific function.