I've got a library written in C++ which I wrap using SWIG and use in python. Generally there is one class with few methods. The problem is that calling these methods may be time consuming - they may hang my application (GIL is not released when calling these methods). So my question is:
What is the simplest way to release GIL for these method calls?
(I understand that if I used a C library I could wrap this with some additional C code, but here I use C++ and classes)
Not having any idea what SWIG is I'll attempt an answer anyway :)
Use something like this to release/acquire the GIL:
class GILReleaser {
GILReleaser() : save(PyEval_SaveThread()) {}
~GILReleaser() {
PyEval_RestoreThread(save);
}
PyThreadState* save;
};
And in the code-block of your choosing, utilize RAII to release/acquire GIL:
{
GILReleaser releaser;
// ... Do stuff ...
}
The real problem is that SWIG is not documented well (I saw hints to use changelog for searching ;) ).
Ok, I found out that I can do inline functions in SWIG and use macros to release/acquire GIL, it looks like this:
%inline %{
void wrappedFunction(OriginalObject *o, <parameters>) {
Py_BEGIN_ALLOW_THREADS
o->originalFunction(<parameters>);
Py_END_ALLOW_THREADS
}
%}
This function is not present in original C++, but available in python module. This is (almost) exactly what I wanted. (what I would like is to wrap original method like python decorator does)
You can use the same API call as for C. No difference. Include "python.h" and call the appoproate function.
Also, see if SWIG doesn't have a typemap or something to indicate that the GIL shuold not be held for a specific function.
Related
Since I am fairly new to Objective-C programming language, I'm facing a huge problem: how can I call a method of my application (made in Objective-C) from my dynamically loaded library (made in C++), by passing it some parameters?
I've been digging the web for a couple of days with no luck. The only thing I found is about IMP, but I'm not sure if that's what I need.
You actually have a plethora of options here.
If the dylib is one of your own, you can compile it as Objective-C++ for Mac OS X and #ifdef the objective-C calls (or not if you are only targeting Mac OS)
Both Obj-C and C++ can make use of C interfaces, so you could write an adapter layer in C (remember Obj-c is a strict superset of C) and expose it for the dylib to call the C functions which then call the Obj-C code.
You can use Blocks, which work in C, C++, and of course Obj-C
you can include the objective-c runtime (see documentation) and muck with that (This is where you would use the *IMP thing you mentioned).
Yet another option might be to use Objective C++ from the Cocoa side to setup C++ objects and expose those to the dylib. How you would go about this really depends on what the library is and how it is used etc; we need to know more about the dylib and how it is being used to elaborate on this.
Since you specifically mention using an IMP lets talk a bit more in depth about that. The declaration is typedef void (*IMP)(id self, SEL _cmd, ...); which you can see takes a pointer to an Obj-C objects, and a SEL (selector), which is just a special C-String representation of the method. You can read more about both SEL and IMP in the documentation.
You can also make use of the runtime's C functions such as objc_msgSend to call a method by passing a pointer to the object and a SEL just like with IMP.
This should be enough information to get you started. Thanks for this question BTW, I never really sat down and thought about all the possible ways to combine C++ with Objective-C before. Odds are I even missed something ;)
You can use objective c runtime
include <objc/runtime.h>
objc_msgSend(id, SEL, arg0, ...)
where
id - is the object where you want to send message
SEL - is struct pointer, describing message you send.
arg0,... are the arguments that you pass to selector.
For more understanding of runtime, see the source code http://www.opensource.apple.com/source/objc4/
Also you can cast IMP address, and call the function.
int(* foo)(id, SEL, arg) = IMP;
foo(someObject, #selector(someMessage), arg);
As I understand the relationship between C and C++, the latter is essentially an extension of the former and retains a certain degree of backwards compatibility. Is it safe to assume that the python C API can be called with C++ code?
More to the point, I notice that the official python documentation bundles C and C++ extensions together on the same page. Nowhere am I able to find a C++ API. This leads me to believe that the same API is safe to use in both languages.
Can someone confirm or deny this?
EDIT:
I think I made my question much more complicated than it needs to be. The question is this: what must I do in order to write a python module in C++? Do I just follow the same directions as listed here, substituting C code for C++? Is there a separate API?
I can confirm that the same Python C API is safe to be used in both languages, C and C++.
However, it is difficult to provide you with more detailed answer, unless you will ask more specific question. There are numerous caveats and issues you should be aware of. For example, your Python extensions are defined as C types struct, not as C++, so don't expect to have their constructor/destructor implicitly defined and called.
For example, taking the sample code from Defining New Types in the Python manual, it can be written in C++ way and you can even blend-in C++ types:
// noddy.cpp
namespace {
struct noddy_NoddyObject
{
PyObject_HEAD
// Type-specific fields go here.
std::shared_ptr<int> value; // WARNING
};
PyObject* Noddy_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
try {
Noddy *self = (Noddy *)type->tp_alloc(type, 0);
if (self) {
self->value = std::make_shared(7);
// or more complex operations that may throw
// or extract complex initialisation as Noddy_init function
return self;
}
}
catch (...) {
// do something, log, etc.
}
return 0;
}
PyTypeObject noddy_NoddyType =
{
PyObject_HEAD_INIT(NULL)
// ...
}
} // unnamed namespace
But, neither constructor nor destructor of the std::shared_ptr will be called.
So, remember to define dealloc function for your noddy_NoddyType where you will reset the value with nullptr. Why even bother with having value defined as shared_ptr, you may ask. It is useful if you use your Python extension in C++, with exceptions, to avoid type conversions and casts, to have more seamless integration inside definitions of your implementation, error handling based on exception may be easier then, etc.
And in spite of the fact that your objects of the noddy_NoddyType are managed by machinery implemented in pure C, thanks to dealloc function the value will be released according to well-known RAII rules.
Here you can find interesting example of nearly seamless integration of Python C API with the C++ language: How To catch Python stdout in c++ code
Python C API can be called within C++ code.
Python C++ extensions are written using the same C API as C extensions use, or using some 3rd party API, such as boost::python.
I have a C++ program:
class X
{
private:
int a;
public:
int func(char *str);
};
Now I want to call the function func in my Python program. How can I do it?
I have successfully called C functions from my Python program using ctypes.
However I can't seem to figure out how to call functions defined inside C++ objects.
I have already worked with ctypes, so I would like to figure out how to accomplishing it that way. However I'm open to other techniques. Also my project puts a constarint that I am not supposed to use wrapper function in my C++ program.
Check out Boost::Python.
You tagged the question ctypes, but you cannot instantiate a C++ object directly from ctypes. You can wrap the C++ object in a C interface and then use ctypes. Or you could write a Python extension module which could include C++ code.
An alternative to Boost.Python is SWIG.
try http://swig.org/
years ago i wrapped some c/c++ code to use them in wxPython with swig.
i cannot remember too much details but i think swig is easy to use.
Hope that helps.
is it possible to wrap a c++ library into c?
how could i do this?
are there any existing tools?
(need to get access to a existing c++ library but only with C)
You can write object-oriented code in C, so if it's an object-oriented C++ library, it's possible to wrap it in a C interface. However, doing so can be very tedious, especially if you need to support inheritance, virtual functions and such stuff.
If the C++ library employs Generic Programming (templates), it might get really hairy (you'd need to provide all needed instances of a template) and quickly approaches the point where it's just not worth doing it.
Assuming it's OO, here's a basic sketch of how you can do OO in C:
C++ class:
class cpp {
public:
cpp(int i);
void f();
};
C interface:
#ifdef __cplusplus
extern "C" {
#endif
typedef void* c_handle;
c_handle c_create(int i)
{
return new cpp(i);
}
void c_f(c_handle hdl)
{
static_cast<cpp*>(hdl)->f();
}
void c_destroy(c_handle hdl)
{
delete static_cast<cpp*>(hdl);
}
#ifdef __cplusplus
}
#endif
Depending on your requirements, you could amend that. For example, if this is going to be a public C interface to a private C++ API, handing out real pointers as handles might make it vulnerable. In that case you would hand out handles that are, essentially, integers, store the pointers in a handle-to-pointer map, and replace the cast by a lookup.
Having functions returning strings and other dynamically sized resources can also become quite elaborate. You would need the C caller provide the buffer, but it can't know the size before-hand. Some APIs (like parts of the WIn32 API) then allow the caller to call such a function with a buffer of the length 0, in which case they return the length of the buffer required. Doing so, however, can make calling through the API horribly inefficient. (If you only know the length of the required buffer after the algorithm executed, it needs to be executed twice.)
One thing I've done in the past is to hand out handles (similar to the handle in the above code) to internally stored strings and provide an API to ask for the required buffer size, retrieve the string providing the buffer, and destroy the handle (which deletes the internally stored string).
That's a real PITA to use, but such is C.
Write a c++ wrapper that does an extern c, compile that with c++, and call your wrapper.
(don't “extern c”)
extern C only helps you to have a names in dll like you see them.
You can use
dumpbin /EXPORTS your.dll
to see what happens with names with extern C or without it.
http://msdn.microsoft.com/en-us/library/c1h23y6c(v=vs.71).aspx
To answer your question... It depends... But it is highly unlikely that you can use it without wrappings. If this C++ library uses just a simple functions and types you can just use it. If this C++ library uses a complex classes structure - probably you will be unable to use it from C without wrapping. It is because the internal of classes may be structured one way or another depending on many conditions (using inference with virtual tables or abstracting. Or in example complex C++ library may have its own object creation mechanisms so you HAVE to use it in the way it is designed or you will get unpredictable behavior).
So, I think, you have to prepare yourself for doing dome wrappings.
And here is a good article about wrapping C++ classes. It the article the Author tells about wrapping C++ classes to C# but he uses C at first step.
http://www.codeproject.com/KB/cs/marshalCPPclass.aspx
If the C++ library is written which can be compiled with C compiler with slight editting (such as changing bool to int, false to 0 and true to 1 etc), then that can be done.
But not all C++ code can be wrapped in C. Template is one feature in C++ that cannot be wrapped, or its nearly impossible.
Wrap it in C++ cpp that calls that dll, and "extern C" in that file you made.
I recently discovered the existence of boost-python and was astonished by it's apparent simplicity. I wanted to give it a try and started to wrap an existing C++ library.
While wrapping the basic library API calls is quite simple (nothing special, just regular function calls and very common parameters), I don't know how to properly wrap the initialization/cleanup functions:
As it stands, my C++ library requires the caller to first call mylib::initialize() when the program starts, and to call mylib::cleanup() before it ends (actually there is also an initializer object that takes care of that, but it is probably irrelevant).
How should I wrap this using boost python ?
Forcing a Python user to call mymodule.initialize() and mymodule.cleanup() seems not very pythonic. Is there any way to that in an automatic fashion ? Ideally, the call to initialize() would be done transparently when the module is imported and the call to cleanup() also done when the python script ends.
Is there any way to do that ? If not, what is the most elegant solution ?
Thank you.
You could try to do a guard object and assign it to a hidden attribute of your module.
struct MyLibGuard
{
MyLibGuard() { myLib::initialize();}
~MyLibGuard() { myLib::cleanup();}
};
using namespace boost::python;
BOOST_PYTHON_MODULE(arch_lib)
{
boost::shared_ptr<MyLibGuard> libGuard = new MyLibGuard();
class_<MyLibGuard, boost::shared_ptr<MyLibGuard>, boost::noncopyable>("MyLibGuard", no_init);
scope().attr("__libguard") = libGuard;
}