This question already has answers here:
Integrate Python And C++
(11 answers)
Closed 3 years ago.
I'm looking for a way to run C++ code alongside python in real time. I have a neural network program which takes in inputs, and outputs a certain integer. The process of running the neural network is the main "meat" of my program and uses up the most resources so I'd like to code it in c++ to make the overall system more efficient.
What I'm looking to do now if run a python main program that passes in python variables to an external CPP file. The CPP file does its math magic and spits put an integer in Python. Is this possible? Is it advisable?
I have heard of Cython before but would much rather code this by scratch to ensure it is optimized for my specific task.
Kindest reguards!
I did something like you want, but with C++ as start point:
C++ code creates python-like module with C functions.
After that C++ code runs target python module, that invokes functions from python-like module.
As result, python module is running and is invoking C++ functions.
Here the sample of C++ code:
// Sample of C++ function, which python module invokes
static PyObject* OutDtmf(PyObject* self, PyObject* args)
{
PyObject* a1;
if (PyArg_UnpackTuple(args, "OutDtmf", 1, 1, &a1))
{
// Check the result
PyObject* astr = PyUnicode_AsUTF8String(a1);
const char* ustr = PyBytes_AsString(astr);
OutDtmf(ustr);
}
Py_RETURN_NONE;
}
// Pack of definitions
// --------------------
static PyMethodDef WarSysMethods[] = {
{ "Finish", FinishScript, METH_VARARGS, NULL },
{ "KpsoSetControl", KpsoSetControl, METH_VARARGS, NULL },
{ "KpsoSetLine", KpsoSetLine, METH_VARARGS, NULL },
{ "OutDtmf", OutDtmf, METH_VARARGS, NULL },
{ "PsoSetLine", PsoSetLine, METH_VARARGS, NULL},
{ NULL, NULL, 0 , nullptr }
};
static struct PyModuleDef WarSysModuleDef = {
PyModuleDef_HEAD_INIT,
"WarSys",
NULL,
-1,
WarSysMethods };
PyMODINIT_FUNC PyInit_WarSys(void)
{
PyObject *module;
module = PyModule_Create(&WarSysModuleDef);
return module;
}
// Start point for creation of python-like module and loading target python module
void StartScript(bool serverMode, const char* testModuleName)
{
// Initialization Python -> C++
PyImport_AppendInittab("WarSys", PyInit_WarSys);
// Initialization C++ -> Python
Py_Initialize();
PyObject* pDict; // borrowed
TestModule = PyImport_ImportModule(testModuleName);
if (!TestModule)
{
PyErr_Print();
return;
}
pDict = PyModule_GetDict(TestModule);
// Read function objects
FuncInit = PyDict_GetItemString(pDict, "Init");
....................
// Invokes python function in module (f.e. Init)
PyObject_CallObject(command.Function, command.Arguments)
in python code use:
import WarSys
and invokes functions WarSys.Finish(False) or other.
Related
Creating an application in C++, I integrated CPython to facilitate the development of certain top-level logics.
The application has a plugin subsystem, which can loaded/unloaded plugins at runtime, this implies to add and remove Python definitions at runtime.
I found that I can add functions with PyModule_AddFunctions, and similarly, I can add constants, objects, etc...
But I found no equivalent PyModule_RemoveFunction.
How to remove a Python function from a module using C++?
Note: Ideally, I would like to avoid solutions like:
Removing the full module and reloading everything
Crafting Python code, that when executed would remove the function.
Let see an example:
DISCLAIM: I removed most of the error checks for simplicity.
#define PY_SSIZE_T_CLEAN
#include <Python.h> //before std includes
// Just an example function to add/remove from Python
static PyObject* log(PyObject* , PyObject* args)
{
//Do something
Py_INCREF(Py_None);
return Py_None;
}
// Setup a "AppModule" module
PyMODINIT_FUNC initAppModule()
{
static PyModuleDef AppModuleInfo{
PyModuleDef_HEAD_INIT,
"AppModule", // name of module
nullptr, // module documentation, may be NULL
-1, // size of per-interpreter state of the module, or -1 if the module keeps state in global variables.
nullptr
};
const auto AppModule = PyModule_Create(&AppModuleInfo);
PyModule_AddFunctions(AppModule, AppModuleMethods);
return AppModule;
}
// Adding a function when a plugin is loaded
void PluginAddingAFunction()
{
static PyMethodDef AppModuleMethods[]{
{"log", log, METH_VARARGS,
"Log a message in the standard output."},
{NULL, NULL, 0, NULL} // Sentinel
};
PyObject *modName = PyUnicode_FromString("AppModule");
PyObject *mod = PyImport_Import(modName);
PyModule_AddFunctions(mod, AppModuleMethods);
}
// Removing the function when the plugin is unloaded
void PluginRemoveAFunction()
{
PyObject *modName = PyUnicode_FromString("AppModule");
PyObject *mod = PyImport_Import(modName);
// How to do this?
//PyModule_RemoveFunctions(mod, "log");
}
int main(int argn, const char* argv[])
{
Py_SetProgramName(argv[0]);
PyImport_AppendInittab("AppModule", &initAppModule);
Py_Initialize();
PyObject *pmodule = PyImport_ImportModule("AppModule");
PluginAddingAFunction(); // <<===== This is done at any time, when loading a plugin
PyRun_SimpleString("import AppModule\n"
"AppModule.log('Hi World')\n"); // <== Example code
PluginRemoveAFunction(); // <<===== This is done when unloading a plugin
Py_FinalizeEx();
}
You can use PyObject_DelAttr C-apis.
int PyObject_DelAttr(PyObject *o, PyObject *attr_name)
Delete attribute named attr_name, for object o. Returns -1 on failure. This is the equivalent of the Python statement del o.attr_name
Reference.
So you could do something like this to remove log function
void PluginRemoveAFunction()
{
PyObject *modName = PyUnicode_FromString("AppModule");
PyObject *mod = PyImport_Import(modName);
PyObject *funcName = PyUnicode_FromString("log");
PyObject_DelAttr(mod, funcName);
}
I am trying to extend Python 3.6 using C++ on windows, following the official documentation and guide. The Anaconda distribution is used with numpy updated to the latest version. In the project python.h and arrayobject.h has been added to the C++ directories, and python36.lib linked, as advised.
For the code, a simple example which is supposed to create an array with the elements 0,1,2,3, when calling the func1 method:
#include <python.h>
#include <arrayobject.h>
static PyObject* fun(PyObject* self, PyObject* args)
{
PyObject *X;
int x[4] = {0,1,2,3};
int dims[1];
dims[0] = 1;
X = PyArray_SimpleNew(1, dims, NPY_INT64, x);
return X;
}
static PyMethodDef MyMethod[] = {
{ "func1", fun, METH_VARARGS,nullptr },
{ nullptr, nullptr, 0, nullptr }
};
static struct PyModuleDef MyModule = {
PyModuleDef_HEAD_INIT,
"MyModule",
NULL,
-1,
MyMethod
};
PyMODINIT_FUNC PyInit_MyModule(void)
{
(void)PyModule_Create(&MyModule);
import_array();
}
The code builds fine. When I take the resulting .pyd file and import it in Spyder, the kernel crashes. Specifically, the import_array(); command seems to cause the crash, as without it the kernel doesn't crash. However, as the documentation notes, the method crashes then. What is the fix?
It was solved by changing the order:
PyMODINIT_FUNC PyInit_MyModule(void)
{
import_array();
return PyModule_Create(&MyModule);
}
All the documentation seems to concern Python 2.X only, which used different initialization. In python 3, the code needs to modified. (There is a small error also in the creation of the array, but I let it be).
I'm currently learning using LUA in C++ code. And I came to a point where I cannot figure out how to use C++ function in LUA's code.
I want to create simple function in c++ and then use it in LUA.
My approach is as follows (taken from some tutorials):
In main.cpp:
void write(const char* str) {
std::cout<<str<<std::endl;
}
static int l_write(lua_State* L) {
const char* str = lua_tostring(L, 1); // get function argument
write(str); // calling C++ function with this argument...
return 0; // nothing to return!
}
int main(){
lua_State* L = luaL_newstate();
luaL_openlibs(L); // load default Lua libs
if (luaL_loadfile(L, "test.lua")) {
std::cout<<"Error loading script"<<std::endl;
}
lua_pushcfunction(L, l_write);
lua_setglobal(L, "write"); // this is how function will be named in Lua
lua_pcall(L, 0, 0, 0); // run script
}
and in test.lua I've got:
write("Hello, world!")
write("The square root of 2 is "..math.sqrt(2))
x = 42
write("We can use variables too, x = "..x)
The problem appears in the very beginning of this code: I cannot even a load script luaL_loadfile(L, "test.lua") returns value 7 (which is as I checked NIME_AGAIN 7 /* temporarily out of resources */).
Everything else works just fine do provided I don't use my custom c++ functions. I can load values from LUAs file normally, can execute function ect.
I suppose that LUA after reading file already compiles it and then finds out name of function that does not exist, namely "write" and returns error in case of reading this file, is it possible? And if so how to resolve this problem, and how to use this feature correctly?
Mhm, guys. It is strange but I did lua_pop(L, 1) once, run and then delete it and now its working just fine O.o
When programming a C++ Node.JS Addon, what is the equivalent of require('./someModule') so that a Module can be loaded for use within a compiled Addon?
I have found this method:
Handle<String> source =
String::New("NameOfLibrary.register(require('./someModule'))");
Handle<Script> script =
Script::Compile(source);
script->Run();
which if used in conjunction with what I asked here would work nicely, but I was wondering if there was a more native way.
You should be able to access the standard module require function in your initialization function. Generally I'd just call it from there since lazy calls to require aren't a good idea since they are synchronous.
static void init (Handle<Object> target, Handle<Object> module) {
HandleScope scope;
Local<Function> require = Local<Function>::Cast(
module->Get(String::NewSymbol("require")));
Local<Value> args[] = {
String::New("./someModule")
};
Local<Value> someModule = require->Call(module, 1, args);
// Do whatever with the module
}
NODE_MODULE(module_file_name, init);
I have a Visual Studio 2008 C++03 application that uses Lua 5.2.1. I would like to extend Lua with a module called "foo", but when I call require("foo") in my Lua script, I get the error:
foo_test.lua:1: module 'foo' not found:
no field package.preload['process']
no file '!\lua\process.lua'
no file '!\lua\process\init.lua'
no file '!\process.lua'
no file '!\process\
My Lua script:
foo.bar()
My lua_foo.h file:
#include <lua.h>
extern "C" int luaopen_foo( lua_State* L );
My lua_foo.cpp file:
#include "lua_foo.h"
#include <lua.hpp>
static int l_bar( lua_State *L )
{
puts( "in bar()" );
return 1;
}
int luaopen_foo( lua_State *L )
{
static const luaL_Reg foo[] = {
{ "bar", l_bar },
{ NULL, NULL }
};
luaL_newlib( L, foo );
return 1;
}
These are compiled in to a static library "lua_foo.lib" which is statically linked to my main Lua executable.
Can anybody help me understand where I'm going wrong? thanks. I would prefer to avoid c++ wrappers (for now) and I do not want to package this library as a separate DLL from the main Lua engine.
EDIT
The issue was in the lua engine code. I added the luaL_requiref per #NicolBolas 's suggestion.
lua_State* L = luaL_newstate();
if( NULL != L )
{
luaL_openlibs( L );
luaL_requiref( token.get(), "foo", luaopen_foo, 1 );
luaL_dofile( L, "foo_test.lua" );
lua_close( L );
}
It's important to understand how the require machinery works and therefore why your code doesn't.
require is designed to look for Lua scripts in the file system and DLLs. Static libraries are not DLLs; indeed, as far as C/C++ is concerned, once you've finished linking, static libraries are no different than compiling those .c/.cpp files into your application directly.
When require finds a DLL with the appropriate name, it loads it and attempts to find a function named luaopen_<modname>, where <modname> is the name of the module. When it does, it will execute this function and store the value it returns in an internal database of loaded modules.
Calling require for a module will return whatever this function returned; if the module has already been loaded, then the return value is pulled from the database and returned directly.
Simply calling luaopen_foo will not do any of this. Indeed, simply calling this function is a bad idea; it is a Lua function and needs to be called as a Lua function (ie: you need to push it onto the Lua stack with lua_pushcfunction and call it with lua_call and so forth).
If you want to create a local module (one not in a Lua script or DLL, but exposed from your code), then you need to use the Lua facilities to do that. Specifically, use luaL_requiref:
luaL_requiref(L, "foo", luaopen_foo, 0);
Call this instead of calling luaopen_foo directly. This will automatically register the return value from luaopen_foo with require's internal database of loaded modules. Thus, subsequent calls to require "foo" will return this table.
One more thing: do is a keyword in Lua; you should not use keywords for Lua table key names. You can, but you always have to quote them (ie: your script must do foo["do"](...) to call it).
luaopen_foo creates a table with one function in it, but it doesn't expose it to Lua in any way. You need to assign it to something your scripts can access if you want to access it. You can do this with the package mechanism, or just assign it to a global (which is what Lua's built-in libraries do).
You have a field named do, which is problematic if you want to use foo.do syntax, because do is a keyword.
The return value of a Lua function tells Lua how many values you left on the stack. Your l_do function lies with its return value.
In the case of luaopen_foo, since you're calling it directly and ignoring it's return value, there's no need for it to return anything at all.
Change your code to this:
static int l_bar( lua_State *L )
{
puts("l_bar called.");
return 0;
}
void luaopen_foo( lua_State *L )
{
static const struct luaL_Reg foo[] = {
{ "bar", l_bar },
{ NULL, NULL }
};
luaL_newlib( L, foo ); // create table containing `bar`
lua_setglobal(L, "foo"); // assign that table to global `foo`
}
And change your script to this:
foo.bar()