We have a DLL that implements a custom programming language. What I want to do is adding support for the python language keeping the same code for "API function".
I have succefully embedded python in this DLL, now I'm approaching the problem to expose all the old function as a python module.
Now this DLL doesn't expose the API function as interface function but it's installed (as function pointer) to the language engine. In this way it's impossible to create a new python module (a new DLL). But I need to keep the compatibility with the old method...
It's possible to create (and install) at runtime a module defined in the same DLL where the Python is located?
I think something like calling the PyInit_xxxx method after PyInitialize();
This has gotten substantially more complicated in Python 3 (vs. how it was in Python 2), but I've gotten it working for my code, so I hope this works for you as well.
// Python 3's init function must return the module's PyObject* made
// with PyModule_Create()
PyObject* initspam();
const char* spam_module_name;
int main(int argc, char **argv)
{
Py_Initialize();
PyImport_AddModule(spam_module_name);
PyObject* module = initspam();
PyObject* sys_modules = PyImport_GetModuleDict();
PyDict_SetItemString(sys_modules, spam_module_name, module);
Py_DECREF(module)
...
}
I found an example of this in the python 3 source code:
Python-3.4.2\Python\pythonrun.c : import_init()
That has much better error checking and such than my example above.
I've solved using a code like this before Py_Initialize();
/* Add a built-in module, before Py_Initialize */
PyImport_AppendInittab("xxx", PyInit_xxx);
The easiest way to handle this is to statically initialize your statically-linked modules by directly calling initspam() after the call to Py_Initialize() or PyMac_Initialize():
int main(int argc, char **argv)
{
/* Pass argv[0] to the Python interpreter */
Py_SetProgramName(argv[0]);
/* Initialize the Python interpreter. Required. */
Py_Initialize();
/* Add a static module */
initspam();
An example may be found in the file Demo/embed/demo.c in the Python source distribution.
Related
I've wrapped a C++ class using Py++ and everything is working great in Python. I can instantiate the c++ class, call methods, etc.
I'm now trying to embed some Python into a C++ application. This is also working fine for the most-part. I can call functions on a Python module, get return values, etc.
The python code I'm calling returns one of the classes that I wrapped:
import _myextension as myext
def run_script(arg):
my_cpp_class = myext.MyClass()
return my_cpp_class
I'm calling this function from C++ like this:
// ... excluding error checking, ref counting, etc. for brevity ...
PyObject *pModule, *pFunc, *pArgs, *pReturnValue;
Py_Initialize();
pModule = PyImport_Import(PyString_FromString("cpp_interface"));
pFunc = PyObject_GetAttrString(pModule, "run_script");
pArgs = PyTuple_New(1); PyTuple_SetItem(pArgs, 0, PyString_FromString("an arg"));
pReturnValue = PyObject_CallObject(pFunc, pArgs);
bp::extract< MyClass& > extractor(pReturnValue); // PROBLEM IS HERE
if (extractor.check()) { // This check is always false
MyClass& cls = extractor();
}
The problem is the extractor never actually extracts/converts the PyObject* to MyClass (i.e. extractor.check() is always false).
According to the docs this is the correct way to extract a wrapped C++ class.
I've tried returning basic data types (ints/floats/dicts) from the Python function and all of them are extracted properly.
Is there something I'm missing? Is there another way to get the data and cast to MyClass?
I found the error. I wasn't linking my bindings in my main executable because the bindings were compiled in a separate project that created the python extension only.
I assumed that by loading the extension using pModule = PyImport_Import(PyString_FromString("cpp_interface")); the bindings would be loaded as well, but this is not the case.
To fix the problem, I simply added the files that contain my boost::python bindings (for me, just wrapper.cpp) to my main project and re-built.
I've been following some code examples on how to build a C module for Python, however it seems like Py_InitModule isn't defined anywhere.
Most sources say it's within the modsupport.h file, but the macro isn't defined there.
I'm using the includes given by the Win32 binaries download, and everything seems in check.
Any suggestions?
Works for me on 2.7.2, Python 2 or 3? For example, for a module called Example:
Python 2:
/* Module entry point Python 2 */
PyMODINIT_FUNC initExample(void)
{
(void) Py_InitModule("Example", ExampleMethods);
}
Python 3:
/* Module entry point Python 3 */
PyMODINIT_FUNC PyInit_Example(void)
{
return PyModule_Create(&Example_module);
}
I'm working on embedding Python in our test suite application. The purpose is to use Python to run several tests scripts to collect data and make a report of tests. Multiple test scripts for one test run can create global variables and functions that can be used in the next script.
The application also provides extension modules that are imported in the embedded interpreter, and are used to exchange some data with the application.
But the user can also make multiple test runs. I don't want to share those globals, imports and the exchanged data between multiple test runs. I have to be sure I restart in a genuine state to control the test environment and get the same results.
How should I reinitialise the interpreter?
I used Py_Initialize() and Py_Finalize(), but get an exception on the second run when initialising a second time the extension modules I provide to the interpreter.
And the documentation warns against using it more than once.
Using sub-interpreters seems to have the same caveats with extension modules initialization.
I suspect that I'm doing something wrong with the initialisation of my extension modules, but I fear that the same problem happens with 3rd party extension modules.
Maybe it's possible to get it to work by launching the interpreter in it's own process, so as to be sure that all the memory is released.
By the way, I'm using boost-python for it, that also warns AGAINST using Py_Finalize!
Any suggestion?
Thanks
Here is another way I found to achieve what I want, start with a clean slate in the interpreter.
I can control the global and local namespaces I use to execute the code:
// get the dictionary from the main module
// Get pointer to main module of python script
object main_module = import("__main__");
// Get dictionary of main module (contains all variables and stuff)
object main_namespace = main_module.attr("__dict__");
// define the dictionaries to use in the interpreter
dict global_namespace;
dict local_namespace;
// add the builtins
global_namespace["__builtins__"] = main_namespace["__builtins__"];
I can then use use the namespaces for execution of code contained in pyCode:
exec( pyCode, global_namespace, lobaca_namespace );
I can clean the namespaces when I want to run a new instance of my test, by cleaning the dictionaries:
// empty the interpreters namespaces
global_namespace.clear();
local_namespace.clear();
// Copy builtins to new global namespace
global_namespace["__builtins__"] = main_namespace["__builtins__"];
Depending at what level I want the execution, I can use global = local
How about using code.IteractiveInterpreter?
Something like this should do it:
#include <boost/python.hpp>
#include <string>
#include <stdexcept>
using namespace boost::python;
std::string GetPythonError()
{
PyObject *ptype = NULL, *pvalue = NULL, *ptraceback = NULL;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
std::string message("");
if(pvalue && PyString_Check(pvalue)) {
message = PyString_AsString(pvalue);
}
return message;
}
// Must be called after Py_Initialize()
void RunInterpreter(std::string codeToRun)
{
object pymodule = object(handle<>(borrowed(PyImport_AddModule("__main__"))));
object pynamespace = pymodule.attr("__dict__");
try {
// Initialize the embedded interpreter
object result = exec( "import code\n"
"__myInterpreter = code.InteractiveConsole() \n",
pynamespace);
// Run the code
str pyCode(codeToRun.c_str());
pynamespace["__myCommand"] = pyCode;
result = eval("__myInterpreter.push(__myCommand)", pynamespace);
} catch(error_already_set) {
throw std::runtime_error(GetPythonError().c_str());
}
}
I'd write another shell script executing the sequence of test scripts with new instances of python each time. Or write it in python like
# run your tests in the process first
# now run the user scripts, each in new process to have virgin env
for script in userScript:
subprocess.call(['python',script])
I want to pass callback from my python code to c++
I want my code look something like this:
In C++ :
typedef void (*MyCallback_t) (CallbackInfo);
class MyClass
{...
void setcallback(MyCallback_t cb);
...
}
And to use it in python :
import mylib
def myCallback(mylib_CallbackInfo):
...
t = mylib.MyClass()
t.setcallback(myCallback)
I saw some topics near my problem but couldn't solve it
For example here :
Realtime processing and callbacks with Python and C++ there is advice to use boost::python and warning about GLI but no examples.
And here
How to call a python function from a foreign language thread (C++) there is no full description with python code part and with "BOOST_PYTHON_MODULE" part
I also found link to use py_boost_function.hpp for example in Boost python howto but it didn't compile and actualy I couldn't understand how to use it.
Ok, I'm still trying to figure this out too, but here's whats working for me so far:
#this is the variable that will hold a reference to the python function
PyObject *py_callback;
#the following function will invoked from python to populate the call back reference
PyObject *set_py_callback(PyObject *callable)
{
py_callback = callable; /* Remember new callback */
return Py_None;
}
...
#Initialize and acquire the global interpreter lock
PyEval_InitThreads();
#Ensure that the current thread is ready to call the Python C API
PyGILState_STATE state = PyGILState_Ensure();
#invoke the python function
boost::python::call<void>(py_callback);
#release the global interpreter lock so other threads can resume execution
PyGILState_Release(state);
The python function is invoked from C++, and executes as expected.
These test files from the boost::python source code repository contains great examples about how to pass callbacks from Python into C++:
https://github.com/boostorg/python/blob/develop/test/callbacks.cpp
https://github.com/boostorg/python/blob/develop/test/callbacks.py
I'm currently writing an applications that embedds the python interpreter. The idea is to have the program call user specified scripts on certain events in the program. I managed this part but now I want the scripts to be able to call functions in my program.
Here's my code so far:
#include "python.h"
static PyObject* myTest(PyObject* self,PyObject *args)
{
return Py_BuildValue("s","123456789");
}
static PyMethodDef myMethods[] = {{"myTest",myTest},{NULL,NULL}};
int main()
{
Py_Initialize();
Py_InitModule("PROGRAM",myMethods);
PyRun_SimpleString("print PROGRAM.myTest()");
Py_Finalize();
}
Thanks!
You need to bind that function to some module, see http://docs.python.org/extending/embedding.html#extending-embedded-python
Edit:
Basicly your code should work. Whats not working?