Calling a Python function within a C++ program - c++

I have a real simple Python function:
def myfunc(x):
return 2.0 * x
I want to send this function to a C++ program and call it so I have done this:
#include "Python.h"
static PyObject *execMyPyFunc(PyObject *self, PyObject *args) {
PyObject *Fx, *pyresult;
double x;
PyArg_ParseTuple(args, "dO", &x, &Fx);
pyresult = PyObject_CallFunction(Fx, "d", x);
return pyresult;
}
static PyMethodDef C_API_TestMethods[] = {
{"execMyPyFunc", execMyPyFunc, METH_VARARGS, "Add documentation here.."},
{NULL, NULL}
};
PyMODINIT_FUNC initC_API_Test(void) {
Py_InitModule("C_API_Test", C_API_TestMethods);
}
My Python program works correctly:
from C_API_Test import execMyPyFunc
def myfunc(x):
return 2.0 * x
fx = execMyPyFunc(1.28,myfunc)
print fx
What I would like to do though is to somehow get the pointer from my Python function (PyObject *Fx) and pass this to a C++ function expecting: double(*fx)(double). Does anyone know how to do this (if possible)? I tried to initialize double(*cFx)(double) and cast my Python function as cFx = (double(*)(double))Fx but this does not work. Any ideas?

You aren't going to be able to simply cast a Python function to C like that.
Instead pass the PyObject function pointer, call the function, and convert to C double. This code will return -1 on failure.
static double cFx(PyObject *fx, double x){
PyObject *pyresult = PyObject_CallFunction(fx, "d", x);
if (pyresult == NULL) return -1;
double cppresult = PyFloat_AsDouble(pyresult);
Py_DECREF(pyresult);
if (PyErr_Occurred()) return -1;
return cppresult;
}
Important part of this is to decrement the reference count to the return value of PyObject_CallFunction since you aren't passing it off to the Python interpreter to deal with.

Related

How to convert PyObject to UTF-8 string?

I'm trying to pass a string value to a Python function and get a utf-8 string value. The function retrieves Japanese characters and returns characters in another language.
I used ctypes.windll.user32.MessageBoxW in the Python script and made sure there is no problem with the Python function. And I tried a simple function like return 'hello world' to test. I also checked every function with assert. I guess the problem starts with PyBytes_AsString. It always returns DB DB DB DB DB DB DB... (in hex). But I don't know how to fix this.
char* result;
PyObject* module, *func, *arg, *ret, *value;
Py_Initialize();
PyObject* sysPath = PySys_GetObject("path");
PyObject* path = PyUnicode_FromString(".");
PyList_Append(sysPath, path);
module = PyImport_ImportModule("test");
if (module != 0)
{
const wchar_t* w = L"翻訳テスト";
func = PyObject_GetAttrString(module, "translate");
arg = PyTuple_New(1);
value = PyUnicode_FromWideChar(w, wcslen(w));
PyTuple_SetItem(arg, 0, value);
ret = PyObject_CallObject(func, arg);
PyObject* repr = PyObject_Repr(ret);
PyObject* str = PyUnicode_AsEncodedString(repr, "utf-8", "strict");
result = PyBytes_AsString(str);
Py_DECREF(repr);
Py_DECREF(str);
Py_DECREF(module);
Py_DECREF(func);
Py_DECREF(arg);
Py_DECREF(ret);
fstream file("text.txt", std::ios::out);
file << result;
file.close();
}
Py_Finalize();
result = PyBytes_AsString(str) returns a pointer to the internal buffer of str, so don't Py_DECREF(str) before you write result to the file.

Python C API and C++ functions

I'm trying to extend the Python interpreter in my C++ program, my problem is as follows.
When I'm trying to call a function, explained in the code below, I get a NameError, from the Python interpreter.
The error is
Traceback (most recent call last):
File "", line 3, in module
NameError: name 'func' is not defined
I used the following code to bind it, according to the Python wiki for version 3.3.2, which I'm using here
double func( int a )
{
return a*a-0.5;
}
static PyObject *TestError;
static PyObject * func_test(PyObject * self, PyObject *args)
{
const int * command;
double sts;
if( !PyArg_ParseTuple(args, "i", &command) )
return NULL;
sts = func( *command );
return PyFloat_FromDouble(sts);
}
static PyMethodDef TestMethods[] = {
{"func", func_test, METH_VARARGS,
"Thing."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
static struct PyModuleDef testmodule = {
PyModuleDef_HEAD_INIT,
"test", /* name of module */
NULL, /* module documentation, may be NULL */
-1, /* size of per-interpreter state of the module,
or -1 if the module keeps state in global variables. */
TestMethods
};
PyMODINIT_FUNC PyInit_test()
{
PyObject *m;
m = PyModule_Create(&testmodule);
if (m == NULL)
return NULL;
TestError = PyErr_NewException("test.error", NULL, NULL);
Py_INCREF(TestError);
PyModule_AddObject(m, "error", TestError);
return m;
}
Then I'm calling PyImport_AppendInittab("test", PyInit_test);
Py_Initialize();, and then I'm trying to run a simple test, with
PyRun_SimpleString("import test\n"
"print('Hi!')\n"
"b = func(5)\n"
"print(b)\n");
Yet, I keep getting the error. Can someone please explain, what am I doing wrong here?
PyRun_SimpleString("import test\n"
"print('Hi!')\n"
"b = test.func(5)\n" # <--
"print(b)\n");
EDIT: Another problem:
int command; // not "int *"
double sts;
if( !PyArg_ParseTuple(args, "i", &command) )
Note that I'd recommend using CFFI if you're not familiar yet with how to write a CPython C extension module.
I agree with all the fixes by Armin Rigo and I would add this one:
PyImport_AppendInittab("test", &PyInit_test);
Pass the address of the function to PyImport_AppendInittab.

Numpy C++: How to iterate over PyArrayObject without a segfault

For me, the following all result in a segfault:
my_array->descr->subarray->shape;
my_array->dimensions;
PyArray_SHAPE(my_array);
PyArray_DIMS(my_array);
PyArray_ITEMSIZE(my_array);
PyArray_NBYTES(my_array);
My function looks like this:
static PyObject* exterior(PyObject* self, PyArrayObject* old_simplices_array)
{//code here
The rest of my cpp file looks like this:
#include "Python.h"
#include "numpy/arrayobject.h"
/* function */
static PyMethodDef compiled_methods[] =
{
{"_exterior",(PyCFunction)exterior , METH_VARARGS},
{NULL, NULL} /* Sentinel */
};
PyMODINIT_FUNC init_alto(void)
{
(void) Py_InitModule("_alto", compiled_methods);
import_array();
}
The python code that passes the array to "exterior" just passes an NxM uint array. That part works. I can access the array's strides and data. I just cannot determine the bounds of iteration. I am working from within sage if that makes any difference.
How am I supposed to iterate over an array without segfaulting? If the answer is obvious, please idiotproof your answer.
For a better idea of what the function looks like, see here.
In the past I have done the following to iterate over a PyArrayObject:
static PyObject *func1(PyObject *self, PyObject *args) {
PyArrayObject *X;
int ndX;
npy_intp *shapeX;
PyArray_Descr *dtype;
NpyIter *iter;
NpyIter_IterNextFunc *iternext;
PyArg_ParseTuple(args, "O!", &PyArray_Type, &X);
ndX = PyArray_NDIM(X);
shapeX = PyArray_SHAPE(X);
dtype = PyArray_DescrFromType(NPY_DOUBLE);
iter = NpyIter_New(X, NPY_ITER_READONLY, NPY_KEEPORDER, NPY_NO_CASTING, dtype);
if (iter==NULL) {
return NULL;
}
iternext = NpyIter_GetIterNext(iter, NULL);
dataptr = (double **) NpyIter_GetDataPtrArray(iter);
do {
cout << **dataptr << endl;
} while (iternext(iter));
NpyIter_Deallocate(iter);
return Py_BuildValue(something);
}
To find out more information check out this link: http://docs.scipy.org/doc/numpy/reference/c-api.iterator.html

Assigning Python function to ctypes pointer variable

I have the following C source that I compile into a DLL:
int (*pfuncExtB)(int a, int b);
int funcB(int a, int b)
{
return funcExtB(a, b);
}
int funcExtB(int a, int b)
{
return pfuncExtB(a, b);
}
What I want to do is to make pfuncExtB "point" to a Python function, so this is what I do in Python:
from ctypes import *
def add(a, b):
return a + b
mutdll = cdll.LoadLibrary("my.dll")
pfuncExtB = (POINTER(CFUNCTYPE(c_int, c_int, c_int))).in_dll(mutdll, 'pfuncExtB')
funcB = mutdll.funcB
funcB.argtypes = [c_int, c_int]
funcB.restype = c_int
pfuncExtB.contents = CFUNCTYPE(c_int, c_int, c_int)(add)
print funcB(3 , 4)
After this I expect that the following call to return 7
print funcB(3, 4)
But I get:
Traceback (most recent call last):
..................
print funcB(3, 4)
WindowsError: exception: access violation reading 0x00000001
So what am I doing wrong here? Is it possible to have a Python function "assigned" to a ctypes pointer-to-function variable?
Edit: After seeing Mark Tolonen's workaround (a set function for the pointer to function variable written in C), I found why it didn't work for me when I tried it.
This does not work:
set(CFUNCTYPE(c_int,c_int,c_int)(add))
print funcB(2, 3)
While this works:
callback = CFUNCTYPE(c_int,c_int,c_int)(add)
set(callback)
print funcB(2, 3)
Where set is a C function that assigns the pointer-to-function argument to the global like in Mark's answer. As he pointed out the answer is lays in the docs:
Important note for callback functions:
Make sure you keep references to CFUNCTYPE() objects as long as they are used from C code. ctypes doesn’t, and if you don’t, they may be garbage collected, crashing your program when a callback is made.
The correct type of the global variable in Python is CFUNCTYPE(c_int,c_int,c_int) (no POINTER()), but I don't see a method to change the value of the variable once you have it. If you can add a set function it can work:
C
typedef int (*FUNC)(int,int);
__declspec(dllexport) FUNC pfuncExtB;
__declspec(dllexport) void set(FUNC f)
{
pfuncExtB = f;
}
int funcExtB(int a, int b)
{
return pfuncExtB(a, b);
}
__declspec(dllexport) int funcB(int a, int b)
{
return funcExtB(a, b);
}
Python
from ctypes import *
FUNC = CFUNCTYPE(c_int,c_int,c_int)
#FUNC
def add(a, b):
return a + b
mutdll = cdll.LoadLibrary('x')
mutdll.set.argtypes = [FUNC]
mutdll.set.restype = None
mutdll.set(add) # set the global variable
pfuncExtB = FUNC.in_dll(mutdll,'pfuncExtB')
print(pfuncExtB(1,2)) # -> 3
funcB = mutdll.funcB
funcB.argtypes = [c_int, c_int]
funcB.restype = c_int
print(funcB(3 , 4)) # -> 7
Note that this does not work:
pfuncExtB = POINTER(FUNC).in_dll(mutdll,'pfuncExtB')
pfuncExtB.contents(1,2) # exception!

calling a method from a python interpreter

per Voo's advice on this thread:
How can I "hook into" Python from C++ when it executes a function? My goal is to profile
I have opened up a new thread for a new question, which is, in C++, how can I initialize a PythonInterpreter and then call a method from it. Specifically, I'd like to be able to call cProfile's methods and also get data from it.
Okay that'll be a bit longer. Note that I pretty much ignore all "usual" error checking - pretty much any python method may return NULL in which case you should handle that gracefully. I show the "unusual" part checking if the given object is callable. Note that PyDECREF fails if the object pointer is NULL, Py_XDECREF does not. And now to the code - there may be a better way to solve all this, but this works fine for me and sadly the documentation is extremely lacking.
C++ code:
#include <Python.h>
static PyThreadState *mainstate;
void initPython(){
PyEval_InitThreads();
Py_Initialize();
mainstate = PyThreadState_Swap(NULL);
PyEval_ReleaseLock();
}
void exitPython(){
PyEval_AcquireLock();
PyThreadState_Swap(mainstate);
Py_Finalize();
}
void callScript() {
PyGILState_STATE gstate = PyGILState_Ensure();
PyObject *pName = PyUnicode_FromString("Startup");
PyObject *pModule = PyImport_Import(pName);
Py_DECREF(pName);
PyObject *pFunc = PyObject_GetAttrString(pModule, "startup");
if (pFunc && PyCallable_Check(pFunc)) {
PyObject *arglist = Py_BuildValue("(u)", "TestScript");
PyObject *result = PyObject_CallObject(pFunc, arglist);
Py_DECREF(arglist);
// Now you have the returned object of the function - do something with it.
// In our case that's None, but you should extend the python scrip to return
// whatever you need - see the profiler API.
Py_DECREF(result);
}
Py_XDECREF(pFunc); // XDECREF does not fail if pointer is NULL.
Py_DECREF(pModule);
PyGILState_Release(gstate);
}
int main() {
printf("Start.\n");
initPython();
callScript();
exitPython();
printf("Exit.\n");
return 0;
}
Your specific script that is always called, change this so that you return all the data you want in a useful manner - at the moment we just use cProfile.run() which just prints some information:
Startup.py
import cProfile
def startup(module_name):
print("Start script")
cProfile.run("import " + module_name)
print("Finished script")
Finally the trivial script that is executed:
TestScript.py
sum = 0
for i in range(10000):
sum += i