How to remove a Python function from a module using C++? - c++

Creating an application in C++, I integrated CPython to facilitate the development of certain top-level logics.
The application has a plugin subsystem, which can loaded/unloaded plugins at runtime, this implies to add and remove Python definitions at runtime.
I found that I can add functions with PyModule_AddFunctions, and similarly, I can add constants, objects, etc...
But I found no equivalent PyModule_RemoveFunction.
How to remove a Python function from a module using C++?
Note: Ideally, I would like to avoid solutions like:
Removing the full module and reloading everything
Crafting Python code, that when executed would remove the function.
Let see an example:
DISCLAIM: I removed most of the error checks for simplicity.
#define PY_SSIZE_T_CLEAN
#include <Python.h> //before std includes
// Just an example function to add/remove from Python
static PyObject* log(PyObject* , PyObject* args)
{
//Do something
Py_INCREF(Py_None);
return Py_None;
}
// Setup a "AppModule" module
PyMODINIT_FUNC initAppModule()
{
static PyModuleDef AppModuleInfo{
PyModuleDef_HEAD_INIT,
"AppModule", // name of module
nullptr, // module documentation, may be NULL
-1, // size of per-interpreter state of the module, or -1 if the module keeps state in global variables.
nullptr
};
const auto AppModule = PyModule_Create(&AppModuleInfo);
PyModule_AddFunctions(AppModule, AppModuleMethods);
return AppModule;
}
// Adding a function when a plugin is loaded
void PluginAddingAFunction()
{
static PyMethodDef AppModuleMethods[]{
{"log", log, METH_VARARGS,
"Log a message in the standard output."},
{NULL, NULL, 0, NULL} // Sentinel
};
PyObject *modName = PyUnicode_FromString("AppModule");
PyObject *mod = PyImport_Import(modName);
PyModule_AddFunctions(mod, AppModuleMethods);
}
// Removing the function when the plugin is unloaded
void PluginRemoveAFunction()
{
PyObject *modName = PyUnicode_FromString("AppModule");
PyObject *mod = PyImport_Import(modName);
// How to do this?
//PyModule_RemoveFunctions(mod, "log");
}
int main(int argn, const char* argv[])
{
Py_SetProgramName(argv[0]);
PyImport_AppendInittab("AppModule", &initAppModule);
Py_Initialize();
PyObject *pmodule = PyImport_ImportModule("AppModule");
PluginAddingAFunction(); // <<===== This is done at any time, when loading a plugin
PyRun_SimpleString("import AppModule\n"
"AppModule.log('Hi World')\n"); // <== Example code
PluginRemoveAFunction(); // <<===== This is done when unloading a plugin
Py_FinalizeEx();
}

You can use PyObject_DelAttr C-apis.
int PyObject_DelAttr(PyObject *o, PyObject *attr_name)
Delete attribute named attr_name, for object o. Returns -1 on failure. This is the equivalent of the Python statement del o.attr_name
Reference.
So you could do something like this to remove log function
void PluginRemoveAFunction()
{
PyObject *modName = PyUnicode_FromString("AppModule");
PyObject *mod = PyImport_Import(modName);
PyObject *funcName = PyUnicode_FromString("log");
PyObject_DelAttr(mod, funcName);
}

Related

Accessing C++ files inside of Python [duplicate]

This question already has answers here:
Integrate Python And C++
(11 answers)
Closed 3 years ago.
I'm looking for a way to run C++ code alongside python in real time. I have a neural network program which takes in inputs, and outputs a certain integer. The process of running the neural network is the main "meat" of my program and uses up the most resources so I'd like to code it in c++ to make the overall system more efficient.
What I'm looking to do now if run a python main program that passes in python variables to an external CPP file. The CPP file does its math magic and spits put an integer in Python. Is this possible? Is it advisable?
I have heard of Cython before but would much rather code this by scratch to ensure it is optimized for my specific task.
Kindest reguards!
I did something like you want, but with C++ as start point:
C++ code creates python-like module with C functions.
After that C++ code runs target python module, that invokes functions from python-like module.
As result, python module is running and is invoking C++ functions.
Here the sample of C++ code:
// Sample of C++ function, which python module invokes
static PyObject* OutDtmf(PyObject* self, PyObject* args)
{
PyObject* a1;
if (PyArg_UnpackTuple(args, "OutDtmf", 1, 1, &a1))
{
// Check the result
PyObject* astr = PyUnicode_AsUTF8String(a1);
const char* ustr = PyBytes_AsString(astr);
OutDtmf(ustr);
}
Py_RETURN_NONE;
}
// Pack of definitions
// --------------------
static PyMethodDef WarSysMethods[] = {
{ "Finish", FinishScript, METH_VARARGS, NULL },
{ "KpsoSetControl", KpsoSetControl, METH_VARARGS, NULL },
{ "KpsoSetLine", KpsoSetLine, METH_VARARGS, NULL },
{ "OutDtmf", OutDtmf, METH_VARARGS, NULL },
{ "PsoSetLine", PsoSetLine, METH_VARARGS, NULL},
{ NULL, NULL, 0 , nullptr }
};
static struct PyModuleDef WarSysModuleDef = {
PyModuleDef_HEAD_INIT,
"WarSys",
NULL,
-1,
WarSysMethods };
PyMODINIT_FUNC PyInit_WarSys(void)
{
PyObject *module;
module = PyModule_Create(&WarSysModuleDef);
return module;
}
// Start point for creation of python-like module and loading target python module
void StartScript(bool serverMode, const char* testModuleName)
{
// Initialization Python -> C++
PyImport_AppendInittab("WarSys", PyInit_WarSys);
// Initialization C++ -> Python
Py_Initialize();
PyObject* pDict; // borrowed
TestModule = PyImport_ImportModule(testModuleName);
if (!TestModule)
{
PyErr_Print();
return;
}
pDict = PyModule_GetDict(TestModule);
// Read function objects
FuncInit = PyDict_GetItemString(pDict, "Init");
....................
// Invokes python function in module (f.e. Init)
PyObject_CallObject(command.Function, command.Arguments)
in python code use:
import WarSys
and invokes functions WarSys.Finish(False) or other.

Extending Python/Numpy using C++, module crashes on initialization

I am trying to extend Python 3.6 using C++ on windows, following the official documentation and guide. The Anaconda distribution is used with numpy updated to the latest version. In the project python.h and arrayobject.h has been added to the C++ directories, and python36.lib linked, as advised.
For the code, a simple example which is supposed to create an array with the elements 0,1,2,3, when calling the func1 method:
#include <python.h>
#include <arrayobject.h>
static PyObject* fun(PyObject* self, PyObject* args)
{
PyObject *X;
int x[4] = {0,1,2,3};
int dims[1];
dims[0] = 1;
X = PyArray_SimpleNew(1, dims, NPY_INT64, x);
return X;
}
static PyMethodDef MyMethod[] = {
{ "func1", fun, METH_VARARGS,nullptr },
{ nullptr, nullptr, 0, nullptr }
};
static struct PyModuleDef MyModule = {
PyModuleDef_HEAD_INIT,
"MyModule",
NULL,
-1,
MyMethod
};
PyMODINIT_FUNC PyInit_MyModule(void)
{
(void)PyModule_Create(&MyModule);
import_array();
}
The code builds fine. When I take the resulting .pyd file and import it in Spyder, the kernel crashes. Specifically, the import_array(); command seems to cause the crash, as without it the kernel doesn't crash. However, as the documentation notes, the method crashes then. What is the fix?
It was solved by changing the order:
PyMODINIT_FUNC PyInit_MyModule(void)
{
import_array();
return PyModule_Create(&MyModule);
}
All the documentation seems to concern Python 2.X only, which used different initialization. In python 3, the code needs to modified. (There is a small error also in the creation of the array, but I let it be).

Explicit DLL 1 function failing

I am writing the following code
#include "SwiWrapper.h"
#include <windows.h>
HINSTANCE hDLL;
SwiWrapper::SwiWrapper()
{
}
SwiWrapper::~SwiWrapper()
{
}
bool SwiWrapper::Initialize()
{ // Handle to DLL
hDLL = LoadLibrary("SWI32.dll");
return true;
}
void SwiWrapper::CloseDll()
{
FreeLibrary(hDLL);
}
//WiRawImage* CALLSPEC WiCreateRawImage ARGSPEC((void));
typedef WiRawImage*(*FuncWiCreateRawImage) (void);
WiRawImage * SwiWrapper::WiCreateRawImage()
{
FuncWiCreateRawImage Exec = (FuncWiCreateRawImage)GetProcAddress(hDLL, "WiCreateRawImage");
return Exec();
}
//WiCmpImage* CALLSPEC WiCreateCmpImage ARGSPEC((void));
typedef WiCmpImage*(*FuncWiCreateCmpImage) (void);
WiCmpImage * SwiWrapper::WiCreateCmpImage()
{
FuncWiCreateCmpImage Exec = (FuncWiCreateCmpImage)GetProcAddress(hDLL, "WiCreateCmpImage");
return Exec();
}
// WiDecmpOptions* CALLSPEC WiCreateDecmpOptions ARGSPEC((void));
typedef WiDecmpOptions*(*FuncWiCreateDecmpOptions) (void);
WiDecmpOptions * SwiWrapper::WiCreateDecmpOptions()
{
FuncWiCreateDecmpOptions Exec = (FuncWiCreateDecmpOptions)GetProcAddress(hDLL, "WiCreateDecmpOptions");
return Exec();
}
//int CALLSPEC WiDecompress ARGSPEC(( WiDecmpOptions *DecmpOptions, WiRawImage *RawImage, WiCmpImage *CmpImage ));
typedef int(*WiDecompressFunc) (WiDecmpOptions*, WiRawImage*, WiCmpImage*);
int SwiWrapper::WiDecompress(WiDecmpOptions * DecmpOptions, WiRawImage * RawImage, WiCmpImage * CmpImage)
{
WiDecompressFunc Exec = (WiDecompressFunc)GetProcAddress(hDLL, "WiDecompress");
int result = Exec(DecmpOptions, RawImage, CmpImage);
return 0;
}
And I am using it like this
SwiWrapper *wrapper = new SwiWrapper();
if (initialized)
{
image = wrapper->WiCreateRawImage();
cmpImage = wrapper->WiCreateCmpImage();
decmpOpts = wrapper->WiCreateDecmpOptions();
GetCmpImage(cmpImage, "path\\data.bin");
SetDecompressionOptions(decmpOpts);
wrapper->WiDecompress(decmpOpts, image, cmpImage); //This line is failing
FileImage("path\\data.jpg", image);
wrapper->CloseDll();
}
However when I reach the following line wrapper->WiDecompress(decmpOpts, image, cmpImage); it fails and gives the following error
Run-Time Check Failure #0 - The value of ESP was not properly saved
across a function call. This is usually a result of calling a
function declared with one calling convention with a function pointer
declared with a different calling convention.
I think that I am messing up with the parameters but I am not sure what I am doing wrong. I have the header file to know the input and outputs.
I am not that experienced with C++, well not lately, so my searching didn't get me to an answer that worked although I found a few Stackoverflow solutions on the error but I cannot figure out what I need to change
Everything that I have on the swi32.dll is in this rar file in this dropbox link
https://www.dropbox.com/s/2bfhylzb2evrggp/Lib.rar?dl=0
My full source code is in the following link
https://www.dropbox.com/s/jkxfyt6xjeanvng/ConsoleApplication1.rar?dl=0
Because all other function don't take an argument, I suppose that the CALLSPEC that is mentioned in the comment for the Decompress function is different to the calling convention you use.
Recheck the calling conventions and the definition of your typedef for the Decompress function.
I have managed to get it working by changing
this line
typedef int(*WiDecompressFunc) (WiDecmpOptions*, WiRawImage*, WiCmpImage*);
to this line
typedef int(__stdcall *WiDecompressFunc)(WiDecmpOptions * DecmpOptions, WiRawImage * RawImage, WiCmpImage * CmpImage);
Credits to:
#Mgetz, #HansPassant who pointed out to use __stdcall and also the link how to use it

How to implement async Constructor/Factory-Call in native C++ / nodejs addon?

i'm quite new to C++ and node/v8 Addon development.
I'm trying to wrap a 3rd Party C-Library.
Some initialisation-functions run quite long and I'd like to run those operations async (with help of libuv).
Given i have the following code:
#define BUILDING_NODE_EXTENSION
#include <node.h>
#include "myobject.h"
//header files of 3rd party lib to be wrapped
#include "3rdparty.h"
using namespace v8;
MyObject::MyObject(void* base) : baseobject_(base) {};
MyObject::~MyObject() {};
void MyObject::Init(Handle<Object> target) {
// Prepare constructor template
Local<FunctionTemplate> tpl = FunctionTemplate::New(New);
tpl->SetClassName(String::NewSymbol("MyObject"));
tpl->InstanceTemplate()->SetInternalFieldCount(1);
Persistent<Function> constructor = Persistent<Function>::New(tpl->GetFunction());
target->Set(String::NewSymbol("MyObject"), constructor);
}
Handle<Value> MyObject::New(const Arguments& args) {
HandleScope scope;
if (args.IsConstructCall())
{
void* cStruct = NULL;
//
//VERY LONG TAKING OPERATION
//Want to run this async with help of libuv
createStructIn3rdPartyLib(&cStruct);
//create actual Object and pass in cStruct which becomes private field.
MyObject* obj = new MyObject(cStruct);
obj->Wrap(args.This());
return args.This();
}
return scope.Close(Undefined());
}
I want to run createStructIn3rdPartyLib(&cStruct); with the help of the libuv-library.
The following is what i came up with. Unfortunately it gives me a Segmentation Fault and I'm not quite sure if it's even the right approach.
I'v already looked into source of other native node-addons, but did not find any solution for my Problem. :(
Any hints welcome.
#define BUILDING_NODE_EXTENSION
#include <node.h>
#include "myobject.h"
//header files of 3rd party lib to be wrapped
#include "3rdparty.h"
using namespace v8;
// libuv allows us to pass around a pointer to an arbitrary
// object when running asynchronous functions. We create a
// data structure to hold the data we need during and after
// the async work.
typedef struct AsyncData {
Persistent<Function> callback; // callback function
void *3rdpartydata
Arguments *args;
} AsyncData;
MyObject::MyObject() {};
MyObject::~MyObject() {};
void MyObject::Init(Handle<Object> target) {
// Prepare constructor template
Local<FunctionTemplate> tpl = FunctionTemplate::New(New);
tpl->SetClassName(String::NewSymbol("MyObject"));
tpl->InstanceTemplate()->SetInternalFieldCount(1);
Persistent<Function> constructor = Persistent<Function>::New(tpl->GetFunction());
target->Set(String::NewSymbol("MyObject"), constructor);
target->Set(String::NewSymbol("createObject"),
FunctionTemplate::New(New)->GetFunction(CreateObjectAsync));
}
Handle<Value> MyObject::New(const Arguments& args) {
HandleScope scope;
if (args.IsConstructCall())
{
void* cStruct = NULL;
//
//VERY LONG TAKING OPERATION
//Want to run this async with help of libuv
createStructIn3rdPartyLib(&cStruct);
//create actual Object and pass in cStruct which becomes private field.
MyObject* obj = new MyObject(&cStruct);
obj->Wrap(args.This());
return args.This();
}
return scope.Close(Undefined());
}
Handle<Value> MyObject::CreateObjectAsync(const Arguments& args)
{
HandleScope scope;
// create an async work token
uv_work_t *req = new uv_work_t;
// assign our data structure that will be passed around
AsyncData *asyncData = new AsyncData();
req->data = asyncData;
// expect a function as the 1st argument
// we create a Persistent reference to it so
// it won't be garbage-collected
asyncData->callback = Persistent<Function>::New(
Local<Function>::Cast(args[0]));
*(asyncData->args) = args;
// pass the work token to libuv to be run when a
// worker-thread is available to
uv_queue_work(
uv_default_loop(),
req, // work token
AsyncWork, // work function
(uv_after_work_cb)AsyncAfter // function to run when complete
);
return scope.Close(Undefined());
}
// Function to execute inside the worker-thread.
// It is not safe to access V8, or V8 data structures
// here, so everything we need for input and output
// should go on our req->data object.
void MyObject::AsyncWork(uv_work_t *req) {
// fetch our data structure
AsyncData *asyncData = (AsyncData *)req->data;
// run 3rd Party Function.
createStructIn3rdPartyLib(&asyncData->3rdpartydata);
}
// Function to execute when the async work is complete
// this function will be run inside the main event loop
// so it is safe to use V8 again
void MyObject::AsyncAfter(uv_work_t *req) {
HandleScope scope;
// fetch our data structure
AsyncData *asyncData = (AsyncData *)req->data;
// create an arguments array for the callback
MyObject *cpBase = new MyObject(asyncData->3rdpartydata);
cpBase->Wrap((*asyncData->args).This());
Handle<Value> argv[] = {Null(),(*asyncData->args).This()};
// surround in a try/catch for safety
TryCatch try_catch;
// execute the callback function
asyncData->callback->Call(Context::GetCurrent()->Global(), 2, argv);
if (try_catch.HasCaught())
node::FatalException(try_catch);
// dispose the Persistent handle so the callback
// function can be garbage-collected
asyncData->callback.Dispose();
// clean up any memory we allocated
delete asyncData;
delete req;
}
Thank you!!!
Edit:
I after reading more examples like the node-ogg bindings i'm thinking more and more that i'm on the wrong track with my approach. Maybe i should stay on a much lower level at the C/C++-Side and implement Object-Functionality on the javascript-side.

Calling Require with a C++ Addon

When programming a C++ Node.JS Addon, what is the equivalent of require('./someModule') so that a Module can be loaded for use within a compiled Addon?
I have found this method:
Handle<String> source =
String::New("NameOfLibrary.register(require('./someModule'))");
Handle<Script> script =
Script::Compile(source);
script->Run();
which if used in conjunction with what I asked here would work nicely, but I was wondering if there was a more native way.
You should be able to access the standard module require function in your initialization function. Generally I'd just call it from there since lazy calls to require aren't a good idea since they are synchronous.
static void init (Handle<Object> target, Handle<Object> module) {
HandleScope scope;
Local<Function> require = Local<Function>::Cast(
module->Get(String::NewSymbol("require")));
Local<Value> args[] = {
String::New("./someModule")
};
Local<Value> someModule = require->Call(module, 1, args);
// Do whatever with the module
}
NODE_MODULE(module_file_name, init);