Call Python function from C/C++ - c++

I am trying to call a simple python function which is defined in ("ctest.py") as
def square(a)
return a**2
the following ("pytest.c") (in same directory) is the C code that I am trying to use to call the function. The issue that I am experiencing is that when the C program tries to load the python module NULL is returned.
#include <Python.h>
int main(int argc, char* argv[])
{
printf("Calling Python Function\n");
Py_Initialize(); // Initialize the Python interpreter.
PyObject *pName, *pModule, *pDict, *pFunc, *pArgs, *pValue; // Create some Python objects that will later be assigned values.
// Convert the file name to a Python string.
pName = PyString_FromString("ctest.py");
if (pName==NULL)
printf("file not found");
else
printf("%s\n", PyString_AsString(pName));
// Import the file as a Python module.
pModule = PyImport_Import(pName); //PROBLEM LINE
if(pModule==NULL)
printf("no Module\n");
// Create a dictionary for the contents of the module.
pDict = PyModule_GetDict(pModule);
printf("After Dictionary retrieval\n");
// Get the add method from the dictionary.
pFunc = PyDict_GetItemString(pDict, "square");
printf("after function retrieval\n");
// Convert 2 to a Python integer.
pValue = PyInt_FromLong(2);
// Call the function with the arguments.
PyObject* pResult = PyObject_CallObject(pFunc, pValue);
// Print a message if calling the method failed.
if(pResult == NULL)
printf("Calling the add method failed.\n");
// Convert the result to a long from a Python object.
long result = PyInt_AsLong(pResult);
// Destroy the Python interpreter.
Py_Finalize();
// Print the result.
printf("The result is %d.\n", result);
return 0;
}
The C Code is built with:
gcc -o pytest -lpythhon2.7 -I/usr/include/python2.7 pytest.c

It looks like you are running into a naming/path issue.
You might have a look at this answer:
Why does PyImport_Import fail to load a module from the current directory?

Related

stack smashing calling python function importing tensorflow c++

I am new to tensorflow as well as including python code in c++, therefore I would apprechiate any tips/comments on the following weird behaviour:
I have a c++ class pythoninterface with headerfile pythoninterface.h:
#include <string>
#include <iostream>
class pythoninterface{
private:
const char* file;
const char* funct;
const char* filepath;
public:
pythoninterface();
~pythoninterface();
void CallFunction();
};
The sourcefile pythoninterface.cpp:
#include <Python.h>
#include <string>
#include <sstream>
#include <vector>
#include "pythoninterface.h"
pythoninterface::pythoninterface(){
file = "TensorflowIncludePy";
funct = "myTestFunction";
filepath = "/path/To/TensorflowIncludePy.py";
}
void pythoninterface::CallFunction(){
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
// Initialize the Python Interpreter
Py_Initialize();
//Set in path where to find the custom python module other than the path where Python's system modules/packages are found.
std::stringstream changepath;
changepath << "import sys; sys.path.insert(0, '" << filepath << "')";
const std::string tmp = changepath.str();
filepath = tmp.c_str();
PyRun_SimpleString (this->filepath);
// Build the name object
pName = PyString_FromString(this->file);
// Load the module object
pModule = PyImport_Import(pName);
if(pModule != NULL) {
// pDict is a borrowed reference
pDict = PyModule_GetDict(pModule);
// pFunc is also a borrowed reference
pFunc = PyDict_GetItemString(pDict, this->funct);
if (PyCallable_Check(pFunc))
{
pValue=Py_BuildValue("()");
printf("pValue is empty!\n");
PyErr_Print();
presult=PyObject_CallObject(pFunc,pValue);
PyErr_Print();
} else
{
PyErr_Print();
}
printf("Result is %d!\n",PyInt_AsLong(presult));
Py_DECREF(pValue);
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
}
else{
std::cout << "Python retuned null pointer, no file!" << std::endl;
}
// Finish the Python Interpreter
Py_Finalize();
}
And the Python File from which the function should be included (TensorflowIncludePy.py):
def myTestFunction():
print 'I am a function without an input!'
gettingStartedTF()
return 42
def gettingStartedTF():
import tensorflow as tf #At this point the error occurs!
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
return 42
Finally in my main function I only create a pythoninterface object p and call the function p.CallFunction(). The communication between the c++ and python code works all right, but when (at runtime) the line import tensorflow as tf is reached, I get a *** stack smashing detected *** error message and the program finishes. Can anyone guess what the problem might be or had a similar issue before?
I know there is a c++ tensorflow API, but I feel more comfortable with using tensorflow in python so I thought this might be the perfect solution for me (apparently it is not...:P)

Embedded Python does not work pointing to Python35.zip with NumPy - how to fix?

Okay here's the basic example from the Python website for a simple runpy.exe to run Python scripts below. It works fine using Visual Studio 2015 on x64 Windows after referencing the Python includes and linking to python35.lib for basic functions (the docs don't mention pyvenv.cfg must be in the EXE directory). However, calling a script that imports NumPy leads to this error ImportError: No module named 'numpy' Failed to load "eig" only when using embedded python35.zip, so how does one include NumPy in an embedded Python EXE? I.e. I want to also "embed" NumPy (as a .zip, directory, .dll, or .pyd etc.). I've tried adding the NumPy includes and also linking to npymath.lib but I get the same import error. I've also dug through some Cython wrapper code but haven't found a solution. Here is the Python embedded sample code:
#include <Python.h>
#include <iostream>
int main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pDict, *pFunc;
PyObject *pArgs, *pValue;
int i;
if (argc < 3) {
fprintf(stderr, "Usage: runpy pythonfile funcname [args]\n");
return 1;
}
Py_SetPath(L"python35.zip"); //this is in the current directory
Py_Initialize();
pName = PyUnicode_DecodeFSDefault(argv[1]);
/* Error checking of pName left out */
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if (pModule != NULL) {
pFunc = PyObject_GetAttrString(pModule, argv[2]);
/* pFunc is a new reference */
if (pFunc && PyCallable_Check(pFunc)) {
pArgs = PyTuple_New(argc - 3);
for (i = 0; i < argc - 3; ++i) {
pValue = PyLong_FromLong(atoi(argv[i + 3]));
if (!pValue) {
Py_DECREF(pArgs);
Py_DECREF(pModule);
fprintf(stderr, "Cannot convert argument\n");
return 1;
}
/* pValue reference stolen here: */
PyTuple_SetItem(pArgs, i, pValue);
}
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
if (pValue != NULL) {
printf("Result of call: %ld\n", PyLong_AsLong(pValue));
Py_DECREF(pValue);
}
else {
Py_DECREF(pFunc);
Py_DECREF(pModule);
PyErr_Print();
fprintf(stderr, "Call failed\n");
return 1;
}
}
else {
if (PyErr_Occurred())
PyErr_Print();
fprintf(stderr, "Cannot find function \"%s\"\n", argv[2]);
}
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}
else {
PyErr_Print();
fprintf(stderr, "Failed to load \"%s\"\n", argv[1]);
return 1;
}
Py_Finalize();
return 0;
}
Embed file is here: https://www.python.org/ftp/python/3.5.2/python-3.5.2-embed-amd64.zip, python35.zip inside the archive. Here is the simple test script (runpy eig eig 10 to test - note if you don't embed Python35.zip and have NumPy / SciPy installed it WILL run):
eig.py
import numpy as np
from scipy import linalg
def eig(a):
c = np.random.rand(a,a)*100
c = np.corrcoef(c)
print('You are taking the eigsh of a ', a, '^2 matrix')
e, f = linalg.eig(c)
return print('Eigvals are: ',np.diag(f))
Anyone know how to fix this issue? Much appreciated.
Update: Here's the compiled version x64 Python 3.5 Windows NumPy SciPy and Pandas with Intel MKL included: https://www.dropbox.com/sh/2smbgen2i9ilf2e/AADI8A3pCAFU-EqNLTbOiUwJa?dl=0
This does not work because numpy is not in the zipfile python35.zip.
The runpy-program sets the path to python35.zip: It is thus the only path in the Pythonpath for this programs exception...
You have to add the parent-folder of your local numpy-folder also to the Pythonpath to make it working.

C's Python API: Calling python function 2X causes Access violation writing location

I'm trying to write some C code that calls some python code to do some data analysis, and I came upon a weird issue. If I call initialize python, call a function, finalize python, and then repeat the same 1 time, I get an access violation writing location error the second time I try to call the function. The following simple code:
#include "stdafx.h"
#include <iostream>
#include "Python.h"
int main()
{
for (int testInc = 0; testInc < 2; testInc++)
{
std::cout << testInc + 1 << std::endl;
PyObject *pName, *pModule, *pFunc, *pValue;
Py_Initialize();
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(\"PathToCode\")");
pName = PyUnicode_DecodeFSDefault("MyModuleName");
pModule = PyImport_Import(pName);
pFunc = PyObject_GetAttrString(pModule, "MyFunctionName");
pValue = PyObject_CallObject(pFunc, NULL);
printf("Result: %s\n", PyBytes_AS_STRING(PyUnicode_AsEncodedString(pValue, "ASCII", "strict")));
Py_DECREF(pName);
Py_DECREF(pModule);
Py_DECREF(pFunc);
Py_DECREF(pValue);
Py_Finalize();
}
return 0;
}
(checks on Py_Object*'s being == NULL omitted for brevity, but they all pass). With the python code being:
def myFunctionName():
import numpy
return "Hi!"
consistently throws the error "Unhandled exception at 0x00007FFE1EBC199C (multiarray.cp35-win_amd64.pyd) in TestApplication.exe: 0xC0000005: Access violation writing location 0x000000000000000A." on the second pass through the for loop, and I'm struggling to figure out why. If I place the initialize and finalize commands outside of the for loop, then this works fine, but my understanding of these commands leads me to believe that this code should be functional. Also, if I omit the "import" command in the python script, my C code also runs fine then, leading me to believe that something weird is happening with the import. Am I misunderstanding something?

Extending Python 3 with C++

I'm trying to extend Python 3 using instructions given here and I'm fairly confident I've followed the instructions correctly so far, but it asks me to include this code:
PyMODINIT_FUNC
PyInit_spam(void)
{
PyObject *m;
m = PyModule_Create(&spammodule);
if (m == NULL)
return NULL;
SpamError = PyErr_NewException("spam.error", NULL, NULL);
Py_INCREF(SpamError);
PyModule_AddObject(m, "error", SpamError);
return m;
}
I'm writing this in MSVC++ 2010 and it's warning me that &spammodule is undefined (the name of the module is spammodule.cpp), but it doesn't define it anywhere in the instructions so I assume that it should recognise it automatically as the name of the module.
The full code is:
#include <Python.h>
#include <iostream>
using namespace std;
static PyObject *SpamError;
int main()
{
cout << "Test" << endl;
system("PAUSE");
return(0);
}
static PyObject *spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return PyLong_FromLong(sts);
}
PyMODINIT_FUNC
PyInit_spam(void)
{
PyObject *m;
m = PyModule_Create(&spammodule);
if (m == NULL)
return NULL;
SpamError = PyErr_NewException("spam.error", NULL, NULL);
Py_INCREF(SpamError);
PyModule_AddObject(m, "error", SpamError);
return m;
}
You're still writing C++, so you still need to declare spammodule somewhere. This is given later on the same page:
static struct PyModuleDef spammodule = {
PyModuleDef_HEAD_INIT,
"spam", /* name of module */
spam_doc, /* module documentation, may be NULL */
-1, /* size of per-interpreter state of the module,
or -1 if the module keeps state in global variables. */
SpamMethods
};
No no no, PyModule_Create() accepts a pointer to the module definition structure and has absolutely nothing to do with the name of the source file.

Python embedded in CPP: how to get data back to CPP

While working on a C++ project, I was looking for a third party library for something that is not my core business. I found a really good library, doing exactly what's needed, but it is written in Python. I decided to experiment with embedding Python code in C++, using the Boost.Python library.
The C++ code looks something like this:
#include <string>
#include <iostream>
#include <boost/python.hpp>
using namespace boost::python;
int main(int, char **)
{
Py_Initialize();
try
{
object module((handle<>(borrowed(PyImport_AddModule("__main__")))));
object name_space = module.attr("__dict__");
object ignored = exec("from myModule import MyFunc\n"
"MyFunc(\"some_arg\")\n",
name_space);
std::string res = extract<std::string>(name_space["result"]);
}
catch (error_already_set)
{
PyErr_Print();
}
Py_Finalize();
return 0;
}
A (very) simplified version of the Python code looks like this:
import thirdparty
def MyFunc(some_arg):
result = thirdparty.go()
print result
Now the problem is this:
'MyFunc' executes fine, i can see the print of 'result'.
What i cannot do is read 'result' back from the C++ code. The extract command never finds 'result' in any namespace.
I tried defining 'result' as a global, i even tried returning a tuple, but i cannot get it to work.
First of all, change your function to return the value. printing it will complicate things since you want to get the value back. Suppose your MyModule.py looks like this:
import thirdparty
def MyFunc(some_arg):
result = thirdparty.go()
return result
Now, to do what you want, you have to go beyond basic embedding, as the documentation says. Here is the full code to run your function:
#include <Python.h>
int
main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pFunc;
PyObject *pArgs, *pArg, *pResult;
int i;
Py_Initialize();
pName = PyString_FromString("MyModule.py");
/* Error checking of pName left out as exercise */
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if (pModule != NULL) {
pFunc = PyObject_GetAttrString(pModule, "MyFunc");
/* pFunc is a new reference */
if (pFunc) {
pArgs = PyTuple_New(0);
pArg = PyString_FromString("some parameter")
/* pArg reference stolen here: */
PyTuple_SetItem(pArgs, 0, pArg);
pResult = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
if (pResult != NULL) {
printf("Result of call: %s\n", PyString_AsString(pResult));
Py_DECREF(pResult);
}
else {
Py_DECREF(pFunc);
Py_DECREF(pModule);
PyErr_Print();
fprintf(stderr,"Call failed\n");
return 1;
}
}
else {
if (PyErr_Occurred())
PyErr_Print();
fprintf(stderr, "Cannot find function");
}
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}
else {
PyErr_Print();
fprintf(stderr, "Failed to load module");
return 1;
}
Py_Finalize();
return 0;
}
Based on ΤΖΩΤΖΙΟΥ, Josh and Nosklo's answers i finally got it work using boost.python:
Python:
import thirdparty
def MyFunc(some_arg):
result = thirdparty.go()
return result
C++:
#include <string>
#include <iostream>
#include <boost/python.hpp>
using namespace boost::python;
int main(int, char **)
{
Py_Initialize();
try
{
object module = import("__main__");
object name_space = module.attr("__dict__");
exec_file("MyModule.py", name_space, name_space);
object MyFunc = name_space["MyFunc"];
object result = MyFunc("some_args");
// result is a dictionary
std::string val = extract<std::string>(result["val"]);
}
catch (error_already_set)
{
PyErr_Print();
}
Py_Finalize();
return 0;
}
Some important points:
I changed 'exec' to 'exec_file' out of
convenience, it also works with
plain 'exec'.
The main reason it failed is that i
did not pass a "local" name_sapce to
'exec' or 'exec_file' - this is now
fixed by passing name_space twice.
If the python function returns
unicode strings, they are not
convertible to 'std::string', so i
had to suffix all python strings
with '.encode('ASCII', 'ignore')'.
I think what you need is either PyObject_CallObject(<py function>, <args>), which returns the return value of the function you call as a PyObject, or PyRun_String(<expression>, Py_eval_input, <globals>, <locals>) which evaluates a single expression and returns its result.
You should be able to return the result from MyFunc, which would then end up in the variable you are currently calling "ignored". This eliminates the need to access it in any other way.