As I'm still new to this, I'm facing some problems, here's my C++ code:
#include <python.h>
#define DLLEXPORT extern "C" __declspec(dllexport)
DLLEXPORT PyObject *Add(PyObject *pSelf, PyObject *pArgs)
{
int s,d;
if(!PyArg_ParseTuple(pArgs, "ii" , &s, &d))
{
PyErr_SetString(PyExc_TypeError,
"Add() invalid parameter");
return NULL;
}
return Py_BuildValue("i", s + d);
}
And the Python code:
import ctypes
MyDll = ctypes.cdll.LoadLibrary(r"PyToCppTest.dll")
jj = MyDll.Add(1,2)
I get an error when I run the above Python code:
OSError: exception: access violation reading 0x000000000000000A
I want to pass the data, without converting it, from Python to C++, then convert it inside C++.
Use either an extension or ctypes; you're not supposed to call your extension through ctypes. The point of extensions is to be able to create modules that look native to people using them from Python. ctypes serves to call C code that was written completely oblivious of Python.
There are a few things that are wrong with your code. First and foremost, the proper include is:
#include <Python.h>
Note the capital P. You're probably on Windows, but this wouldn't work on Linux without the capital P.
Also, I don't see the point of the *pSelf pointer in your function declaration, you should get rid of it:
PyObject *Add(PyObject *pArgs)
Now, your main problem is this:
MyDll.Add(1,2)
...does not call MyDll.Add with a tuple. It calls it with two integer arguments, 1 and 2. If you want to pass a tuple, you'd do:
MyDll.Add((1,2))
However, Python's ctypes won't know what to do with this (it normally accepts integer arguments), so you'll need to tell it that Add actually wants a tuple, and returns a Python object:
import ctypes
MyDll = ctypes.cdll.LoadLibrary("PyToCppTest.dll")
MyCFunc = ctypes.PYFUNCTYPE(
ctypes.py_object, # return val: a python object
ctypes.py_object # argument 1: a tuple
)
MyFunc = MyCFunc(('Add', MyDll))
jj = MyFunc((1,2))
Related
I am embedding python in my C++ application using boost python. I am a C++ programmer, with very limited knowledge of Python.
I have a C++ class, PyExpression. Each instance of this class has a string expStr, which is a short user-entered (at runtime) python program, that is executed by calling boost::python::exec. Briefly, I have this set up as:
//import main and its globals
bp::object main = bp::import("__main__");
bp::object main_namespace = main.attr("__dict__");
where main and main_namespace are members of the C++ class PyExpression.
void PyExpression::Run()
{
bp::object pyrun = exec(expStr,main_namespace);
}
The problem here is that different C++ instances of PyExpression modify the same global python namespace, main_namespace, and I want each PyExpression instance to have its own "global" namespace.
If I pass in boost::python::dict class_dict instead of main_namespace above, it works at a basic level. But if PyExpression::expStr imports a module, e.g. import sys, then I get an ImportError. Also, using class_dict, I can no longer call globals(), locals(), vars(), as they all become undefined.
I have also tried exposing PyExpression as a python module. Briefly,
BOOST_PYTHON_MODULE(PyExpModule)
{
bp::class_<PyExpression>("PyExpression", bp::no_init)
//a couple .def functions
}
int pyImport = PyImport_AppendInittab( "PyExpModule", &initPyExpModule );
bp::object thisExpModule = bp::object( (bp::handle<>(PyImport_ImportModule("PyExpModule"))) );
bp::object PyExp_namespace = thisExpModule.attr("__dict__");
Unfortunately, using PyExp_namespace, again I get the ImportError when the string to be executed imports a python module, and again, the namespace is shared between all instances of PyExpression.
In short, I want to be able to use a namespace object/dictionary, that is preferably a class member of PyExpression, have only that instance of PyExpression have access to the namespace, and the namespace to act like a global namespace such that other modules can be imported, and the `globals(), locals(), vars() are all defined.
If anyone can point me to a sketch of working code, I would very much appreciate it. I can't find relevant material on this problem.
Before providing a solution, I want to provide some clarification on Python behavior.
Boost.Python's object is essentially a higher-level handle of a smart pointer. Thus, multiple object instances may point to the same Python object.
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
The above code imports a module named __main__. In Python, modules are essentially singletons due to the import behavior. Therefore, although the C++ main_module may be a member of the C++ PyExpression class, they all point to the same Python __main__ module, as it is a singleton. This results in main_namespace pointing to the same namespace.
Much of Python is built around dictionaries. For example, with an example module:
class Foo:
def __init__(self):
self.x = 42;
def bar(self):
pass
There are 3 dictionaries of interests:
example.__dict__ is the example module's namespace.
>>> example.__dict__.keys()
['__builtins__', '__file__', '__package__', '__name__', 'Foo', '__doc__']
example.Foo.__dict__ is a dictionary that describes the Foo class. Additionally, it would contain the equivalent of C++'s static member variables and functions.
>>> example.Foo.__dict__.keys()
['__module__', 'bar', '__doc__', '__init__']
example.Foo().__dict__ is a dictionary containing instance-specific variables. This would contain the equivalent of C++'s non-static member variables.
>>> example.Foo().__dict__.keys()
['x']
The Python exec statement takes two optional arguments:
The first argument specifies the dictionary that will be used for globals(). If the second argument is omitted, then it is also used for locals().
The second argument specifies the dictionary that will be used for locals(). Variable changes occurring within exec are applied to locals().
To get the desired behavior, example.Foo().__dict__ needs to be used as locals(). Unfortunately, this becomes slightly more complicated because of the following two factors:
Although import is a Python keyword, the CPython implementation is dependent on __builtins__.__import__. Thus, there needs to be a guarantee that the __builtin__ module is assessable as __builtins__ within the namespace passed to exec.
If a C++ class called Foo is exposed as a Python class through Boost.Python, then there is no easy way to access the Python Foo instance from within the C++ Foo instance.
To account for these behaviors, the C++ code will need to:
Get a handle to the Python object's __dict__.
Inject the __builtin__ module into the Python object's __dict__.
Extract the C++ object from the Python object.
Pass the Python object's __dict__ to the C++ object.
Here is an example solution that only sets variables on the instance for which code is being evaluated:
#include <boost/python.hpp>
class PyExpression
{
public:
void run(boost::python::object dict) const
{
exec(exp_.c_str(), dict);
}
std::string exp_;
};
void PyExpression_run(boost::python::object self)
{
// Get a handle to the Python object's __dict__.
namespace python = boost::python;
python::object self_dict = self.attr("__dict__");
// Inject the __builtin__ module into the Python object's __dict__.
self_dict["__builtins__"] = python::import("__builtin__");
// Extract the C++ object from the Python object.
PyExpression& py_expression = boost::python::extract<PyExpression&>(self);
// Pass the Python object's `__dict__` to the C++ object.
py_expression.run(self_dict);
}
BOOST_PYTHON_MODULE(PyExpModule)
{
namespace python = boost::python;
python::class_<PyExpression>("PyExpression")
.def("run", &PyExpression_run)
.add_property("exp", &PyExpression::exp_, &PyExpression::exp_)
;
}
// Helper function to check if an object has an attribute.
bool hasattr(const boost::python::object& obj,
const std::string& name)
{
return PyObject_HasAttrString(obj.ptr(), name.c_str());
}
int main()
{
PyImport_AppendInittab("PyExpModule", &initPyExpModule);
Py_Initialize();
namespace python = boost::python;
try
{
// python: import PyExpModule
python::object py_exp_module = python::import("PyExpModule");
// python: exp1 = PyExpModule.PyExpression()
// python: exp1.exp = "import time; x = time.localtime().tm_year"
python::object exp1 = py_exp_module.attr("PyExpression")();
exp1.attr("exp") =
"import time;"
"x = time.localtime().tm_year"
;
// python: exp2 = PyExpModule.PyExpression()
// python: exp2.exp = "import time; x = time.localtime().tm_mon"
python::object exp2 = py_exp_module.attr("PyExpression")();
exp2.attr("exp") =
"import time;"
"x = time.localtime().tm_mon"
;
// Verify neither exp1 nor exp2 has an x variable.
assert(!hasattr(exp1, "x"));
assert(!hasattr(exp2, "x"));
// python: exp1.run()
// python: exp2.run()
exp1.attr("run")();
exp2.attr("run")();
// Verify exp1 and exp2 contain an x variable.
assert(hasattr(exp1, "x"));
assert(hasattr(exp2, "x"));
// python: print exp1.x
// python: print exp2.x
std::cout << python::extract<int>(exp1.attr("x"))
<< "\n" << python::extract<int>(exp2.attr("x"))
<< std::endl;
}
catch (python::error_already_set&)
{
PyErr_Print();
}
}
And the output:
[twsansbury#localhost]$ ./a.out
2013
5
Due to how libraries are loaded from imports, it may require providing arguments to the linker that will cause all symbols, not only used ones, to the dynamic symbol table. For example, when compiling the above example with gcc, using -rdynamic was required. Otherwise, import time will fail due to an undefined PyExc_IOError symbol.
Python does not provide a 100% reliable isolation mechanism for this kind of task. That said, the essential tool you are looking for is the Python C-API Py_NewInterpreter, which is documented here. You will have to call it upon the creation of your PyExpression object, to create a new (semi)-isolated environment (N.B.: the destructor should call Py_EndInterpreter).
This is untested, but I'd guess something liket this would do the job:
PyThreadState* current_interpreter = Py_NewInterpreter();
bp::object pyrun = exec(expStr);
Py_EndInterpreter(current_interpreter);
You may wrap that into an object. If you wish to do so, you must manage the "thread" state as explained in this other stackoverflow thread.
I am a beginner in python and have an intermediate level knowledge about C++.
I am trying to embed python code in c++. However, I get a build error and with my current level of knowledge I am unable to troubleshoot it.Kindly help me in this regard.
Following is the code.
#include <iostream>
using namespace std;
#include <Python.h>
int main()
{
cout<<"Calling Python to find the sum of 2 and 2";
// Initialize the Python interpreter.
Py_Initialize();
// Create some Python objects that will later be assigned values.
PyObject *pName,*pModule, *pDict, *pFunc, *pArgs, *pValue;
// Convert the file name to a Python string.
pName = PyString_FromString("Sample.py"); // Import the file as a Python module.
pModule = PyImport_Import(pName);
// Create a dictionary for the contents of the module.
pDict = PyModule_GetDict(pModule);
// Get the add method from the dictionary.
pFunc = PyDict_GetItemString(pDict, "add");
// Create a Python tuple to hold the arguments to the method.
pArgs = PyTuple_New(2);
// Convert 2 to a Python integer.
pValue = PyInt_FromLong(2);
// Set the Python int as the first and second arguments to the method.
PyTuple_SetItem(pArgs, 0, pValue);
PyTuple_SetItem(pArgs, 1, pValue);
// Call the function with the arguments.
PyObject* pResult = PyObject_CallObject(pFunc, pArgs);
// Print a message if calling the method failed.
if(pResult == NULL)
cout<<"Calling the add method failed.\n";
// Convert the result to a long from a Python object.
long result = PyInt_AsLong(pResult);
// Destroy the Python interpreter.
Py_Finalize(); // Print the result.
cout<<"The result is"<<result;
cout<<"check";
return 0;
}
I get the following error:
Unhandled exception at 0x00000000 in pytest.exe: 0xC0000005: Access violation.
and the build breaks at the line pModule = PyImport_Import(pName);
the file Sample.py has the contents:
# Returns the sum of two numbers.
def add(a, b):
return a+b
I am using python 2.7 and VS2010.I have created a win32 console project for this and I am building in release mode.I have copied the file Sample.py to the project folder.
I cannot figure out what's causing the build to crash.Kindly help.
Firstly, indent your code, MSVC even has an option to do it automatically! After all, you want people here to read it, so go and clean that up. Then, don't declare variables without initializing them in C++. This makes it clear from where on you can use them. Lastly, whenever you call a function, check its results for errors. By default, just throw std::runtime_error("foo() failed"); on errors. More elaborate, you could try to retrieve and add error information from Python.
Now, your immediate error is the use of a null pointer, which you would have avoided if you had checked returnvalues. After writing the code to correctly detect that error, the next thing I'd look at is the missing initialization of the Python interpreter. You already have a comment in place, but comments don't count. I guess that if you had implemented proper error handling, Python would also have told you about the missing initialization.
For performance reasons I want to port parts of my python program to C++ and I therefore try to write a simple extension for my program. The C++ part will build a dictionary, which then needs to be delivered to the Python program.
One way I found seems to be to build my dict-like object in C++, e.g. a boost::unordered_map, and then translate it to Python using the Py_BuildValue[1] method, which is able to produce Python dicts. But this method which includes converting the container into a string representation and back seems a bit too much 'around the corner' to be the most performant solution!?
So my question is: What is the most performant way to build a Python dictionary in C++? I saw that boost has a Python library which supports mapping containers between C++ and Python, but I didn't find the exact thing I need in the documentation so far. If there is such way I would prefer to directly build a Python dict in C++, so that no copying etc. is needed. But if the most performant way to do this is another one, I'm good with that too.
Here is the (simplified) C++-code I compile into a .dll/.pyd:
#include <iostream>
#include <string>
#include <Python.h>
#include "boost/unordered_map.hpp"
#include "boost/foreach.hpp"
extern "C"{
typedef boost::unordered_map<std::string, int> hashmap;
static PyObject*
_rint(PyObject* self, PyObject* args)
{
hashmap my_hashmap; // DO I NEED THIS?
my_hashmap["a"] = 1; // CAN I RATHER INSERT TO PYTHON DICT DIRECTLY??
BOOST_FOREACH(hashmap::value_type i, my_hashmap) {
// INSERT ELEMENT TO PYTHON DICT
}
// return PYTHON DICT
}
static PyMethodDef TestMethods[] = {
{"rint", _rint, METH_VARARGS, ""},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC
inittest(void)
{
Py_InitModule("test", TestMethods);
}
} // extern "C"
This I want to use in Python like:
import test
new_dict = test.rint()
The dictionary will map strings to integers. Thanks for any help!
Use the CPython API directly yes:
PyObject *d = PyDict_New()
for (...) {
PyDict_SetItem(d, key, val);
}
return d;
Or write a python object that emulate a dict, by overriding __setitem__ and __getitem__. In both method, use your original hashmap. At the end, no copy will happen!
I have a C++ library that has a Python wrapper (written with SWIG). This library allows executing small user-defined code (a callback), such as element-wise operations on a vector. I.e. instead of just a + you can do whatever arbitrary binary function. Right now this is accomplished by accepting a callable Python object for the binary function and calling it. It works, but is about 80 times slower than code that doesn't have to bounce up and down into Python at every iteration.
How would I write/build/import a Cython function could be passed into my C++ library so that it can be called directly by the C++ library?
Edit:
If I just stuck to C then I would write something like
EWise(double (*callback)(double, double))
EWise would then callback(10, 20); or such. I want callback to be written in Cython, using whatever name the user wants, and a pointer to it has to be passed to my C++ library through Python somehow. That somehow is where I'm unclear.
The trick with cython is in using the keyword public
cdef public double cython_function( double value, double value2 ):
return value + value2
Then the command cythonize <your_file.pyx> along with <your_file.c> will create header <your_file.h> that you can include.
Alternatively, you can create the header yourself:
#ifdef __cplusplus {
extern "C"
#endif
double cython_function( double value, double value2 );
#ifdef __cplusplus
}
#endif
Update:
Then with a little overlay from Python you can use ctypes's callback mechanism
func_type = CFUNCTYPE(c_double, c_double, c_double)
your_library.set_callback_function ( func_type(user_modules.cython_function) )
You can achieve that by doing pure cdef functions :
# declare the prototype of your function
ctypedef void (*callback_ptr)(int arg)
# declare your function as cdef
cdef void my_callback(int arg):
print 'doing some python here', arg
# now, you can use the cdef func as a callback
# in pure C !
cdef void run():
cdef callback_ptr p = my_callback
p(42)
if __name__ == '__main__':
run()
Note: you can use "cython -a" to see that they are no python code involved for the content of run. So it will work with your c library.
Embedding Python in Another Application may be useful reading.
I used SWIG to generate a Perl module for a C++ program. I have one function in the C++ code which returns a "char pointer". Now I dont know how to print or get the returned char pointer in Perl.
Sample C code:
char* result() {
return "i want to get this in perl";
}
I want to invoke this function "result" in Perl and print the string.
How to do that?
Regards,
Anandan
Depending on the complexity of the C++ interface, it may be easier, faster, and more maintainable to skip SWIG and write the XS code yourself. XS&C++ is a bit of an arcane art. That's why there is Mattia Barbon's excellent ExtUtils::XSpp module on CPAN. It make wrapping C++ easy (and almost fun).
The ExtUtils::XSpp distribution includes a very simple (and contrived) example of a class that has a string (char*) and an integer member. Here's what the cut-down interface file could look like:
// This will be used to generate the XS MODULE line
%module{Object::WithIntAndString};
// Associate a perl class with a C++ class
%name{Object::WithIntAndString} class IntAndString
{
// can be called in Perl as Object::WithIntAndString->new( ... );
IntAndString();
// Object::WithIntAndString->newIntAndString( ... );
// %name can be used to assign methods a different name in Perl
%name{newIntAndString} IntAndString( const char* str, int arg );
// standard DESTROY method
~IntAndString();
// Will be available from Perl given that the types appear in the typemap
int GetInt();
const char* GetString ();
// SetValue is polymorphic. We want separate methods in Perl
%name{SetString} void SetValue( const char* arg = NULL );
%name{SetInt} void SetValue( int arg );
};
Note that this still requires a valid XS typemap. It's really simple, so I won't add it here, but you can find it in the example distribution linked above.
You must have referred to the SWIG tutorial at www.swig.org/tutorial.html
Anyways, since you just want to invoke the function the C function from perl,
1. Type your interface file(having all the function declarations in the wrapper and the module sections).
2. Compile with swig and options.
3. Compile with gcc to create the objects.
4. Compile with gcc options to create the shared object.
5. run the program as follows:
perl
use moduleName;
$a = moduleName::result();
[NOTE: Look into the generated module file(.pm) for the correct funvtion prototype which points to the correct function in the wrapper file.]