I've started to play around with SWIG, so that I can use C libraries in Python. I have this piece of code where I'm passing a Python String into a C function that expects a "void *"
example.h:
char test(void *buf);
example.c:
char test(void *buf) {
char *c = (char*)buf;
return *c;
}
Python:
import example
buf = "hello"
example.test(buf)
When I run it, I get the following error:
TypeError: in method 'test', argument 1 of type 'void *'
However, when I change the "void*" parameter to "char*", it seems to work. I'm a little confused as I thought "void*" matches any kind of pointer. Anyways, I dug around and came across the ctypes library and casted it to a c_void_p (Python: converting strings for use with ctypes.c_void_p()). That didn't seem to work for me.
As a work around I made a wrapper in my swig file:
/* File: example.i */
%module example
%include typemaps.i
%{
#include "example.h"
%}
%include "example.h"
%inline %{
char test_wrapper(char *buf) {
void *voidBuf = (void*)buf;
return test(voidBuf);
}
%}
This seems to work. However, I was wondering if someone could provide an explanation as to why the ctypes approach didn't work. If I was completely off the mark with the ctypes approach, is there a more appropriate way than creating an inline wrapper?
Thanks guys!
Your inline wrapper approach is a sound generalized way to work around functions that aren't quite usable directly. Generally SWIG tries to mimic the behavior of C or C++ inside the target language. In this case though as you observed it doesn't quite do that. I think the reason for this is twofold: firstly it's not obvious how you'd go about doing that in some of the languages SWIG supports (e.g. Java). Secondly, even though you could do something in Python that would work for this specific case in general void* is pretty ambiguous to a C programmer without further guidance about how it is intended to be interpreted/used. There's a mis-match between C and Python here too - strings are reference counted and immutable in Python so it would be very easy to stray into circumstances where the Python behavior breaks in ways that are totally non-obvious to Python programmers.
Using ctypes to perform the cast isn't really a viable solution, SWIG and ctypes have fundamentally very different approaches to wrapping things and typing of arguments. (If you're curious I just wrote a fairly detailed answer looking at interpoerability between them)
The other option in the general case is to user %include <cpointer.i> to generate some code for casting/converting for you, e.g.:
%pointer_cast(type1, type2, name)
However in this instance since you're using char* so the note in the first note documentation applies:
Note: None of these macros can be used to safely work with strings
(char * or char **).
Note: When working with simple pointers, typemaps can often be used to
provide more seamless operation.
And in general I'd favor the point made by the second note, which is that by doing a little more work (e.g. %inline to supply an overload) gives a more intuitive behavior. So in your specific case the only change I'd recommend would be using %rename(test) test_wrapper; to make it into an overload inside Python. (If it were C++ instead of C that you're targeting then you could do it as an overload inside C++)
I have an old C++ code that has this line of code:
int logprintf( const char* ptr, ... );
and I am not sure what does it do exactly? should I change the code in order to be able to compile the code with new versions of the compiler?! does anybody have an idea? any help would be appreciated.
That's just C code and any compiler made since the 1980s should be able to handle it. The definition is of a varidic function, or "varargs" in C parlance.
What that does is declare a method signature, nothing more. Presumably somewhere else, either in a library you link in or in a C or C++ file you compile there's a matching implementation.
Most printf-style functions do not have a fixed number of arguments. This is what the ellipsis ... represents, zero or more arbitrary arguments go there.
I have a frustrating problem which got me spend a lot of time dealing with it but I did not find any solution.
I want to use C++ class in PHP with SWIG. I generated my shared object and it works fine for some methods but I've got this error whenever I call the methods with string arguments as their input:
Fatal error: Type error in argument 2 of PKI_Buf_initHex. Expected SWIGTYPE_p_std__string
PKI_Buf_initHex is the name of the wrapper class which SWIG made automatically. In my C++ code I declare initHex method as:
int initHex(const string x)
{..}
I included typemaps.i and std_string.i in my interface file but I got the same error message.
I truly would appreciate if anyone can help me with this issue.
You need to have:
%include <std_string.i>
Early enough in the SWIG interface (i.e. before std::string is first seen).
I have a working python wrapper for C++ code (as suggested here Calling C/C++ from python?
) using ctypes. But the problem is with main function of the code. When I do something like
extern "C" {
void call_main(){ main();}
}
in my c++ code and then call this function via python wrapper
...
lib = cdll.lib('./mylib.so')
def run():
lib.call_main()
-> I get "segmentation fault".
The funny part is that when i copy paste my main method code into function called e.g. test (so it is int test() {....#pasted code...} in c++ code), extern it and then call lib.test()
=> And eveything works fine... So it must be a problem with the main function being called main or something
In C++ calling main() recursively is not allowed ( see 3.6.1, basic.start.main, paragraph 3). Also, you need a C++ aware entry point when you want to call C++ functionality. You can sometimes get away with calling C++ functionality without this but what is going to work and what is not isn't entirely straight forward. The obvious problem is with global objects needing initialization.
Just put the code you want to call into a different function and call this.
I have a C++ class. It's made up of one .ccp file and one .h file. It compiles (I can write a main method that uses it successfully in c++). How do I wrap this class with Cython to make it available in Python?
I've read the docs and don't follow. They talk about generating the cpp file. When I've tried to follow the docs, my already existing cpp gets blown away...
What am I meant to put in the pyx file? I've been told the class definition but how much of it? Just the public methods?
Do I need a .pxd file? I don't understand when this file is or isn't required.
I've tried asking these question in the #python IRC channel and can't get an answer.
Even Cython is generally for use with C, it can generate C++ code, too. When compiling, you add the --cplus flag.
Now, creating a wrapper for the class is simple and not much different from wrapping a structure. It mainly differs from declaring the extern, but that's not much difference at all.
Suppose you have a class MyCppClass in mycppclass.h.
cdef extern from "mycppclass.h":
cppclass MyCppClass:
int some_var
MyCppClass(int, char*)
void doStuff(void*)
char* getStuff(int)
cdef class MyClass:
# the public-modifier will make the attribute public for cython,
# not for python. Maybe you need to access the internal C++ object from
# outside of the class. If not, you better declare it as private by just
# leaving out the `private` modifier.
# ---- EDIT ------
# Sorry, this statement is wrong. The `private` modifier would make it available to Python,
# so the following line would cause an error es the Pointer to MyCppClass
# couldn't be converted to a Python object.
#>> cdef public MyCppClass* cobj
# correct is:
cdef MyCppClass* obj
def __init__(self, int some_var, char* some_string):
self.cobj = new MyCppClass(some_var, some_string)
if self.cobj == NULL:
raise MemoryError('Not enough memory.')
def __del__(self):
del self.cobj
property some_var:
def __get__(self):
return self.cobj.some_var
def __set__(self, int var):
self.cobj.some_var = var
Note that the new keyword is only available when the --cplus flag is set, otherwise use malloc from <stdlib.h> by externing it.
Also note that you don't need to dereference the pointer (->) to call the method. Cython tracks the object's type and applies what fits.
.pxd files are for seperating declarations from implementation, or to avoid namespace colliding. Imagine you'd like to name you Python-wrapper like the C++ class. Simply put in your .pxd file the extern declarations and cimport the pxd file in the .pyx.
cimport my_pxd
cdef my_pxd.SomeExternedType obj
Note that you can not write implementations in a .pxd file.
So after lots of poking, trial and error, screaming and tearing my hair out, I finally got this to work. First though, I had to re-write my C++ into C, which for me really just involved converting all my std::string variables to char* and keeping track of some lengths.
Once done I had my .h and .c files. I wanted to make a single function from the C code available in Python. It turns out that Cython can compile your C files into the extension for you and link any libraries all in one go, so starting with my setup.py, it ended up looking like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules=[
Extension("myext",
["myext.pyx", "../stuff.c"],
libraries=["ssl", "crypto"]
)
]
setup(
name = "myext",
cmdclass = {"build_ext": build_ext},
ext_modules = ext_modules
)
As you can see, the second argument to the Extension simply lists all the files that need to be compiled, Cython works out how to compile them depending on their file extension as far as I can tell. The libraries array tells the Cython compiler what needs to be linked in (in this case I was wrapping some crypto stuff that I couldn't seem to mimick directly through existing Python libs).
To actually make my C function available in the .pyx file, you write a small wrapper in the .pxd. My myext.pxd looked as below:
cdef extern from "../stuff.h":
char* myfunc(char* arg1, char* arg2, char* arg3)
In the .pyx you then use the cimport declaration to import this function, which is then available for use as if it were any other Python function:
cimport myext
def my_python_func(arg1, arg2, arg3):
result = myext.myfunc(arg1, arg2, arg3)
return result
When you build this (on Mac at least) you get a .so that you can import in python and run the functions from the .pyx. There may be better, more correct way to get this all working but that comes from experience and this was a first encounter that I managed to work out. I'd be very interested on pointers where I may have gone wrong.
Update:
After further use of Cython, I found it was super simple to integrate it with C++ too, once you know what you're doing. Making C++'s string available is as simple as from libcpp.string cimport string in your pyx/pyd. Declaring the C++ class is similarly easy as:
cdef extern from "MyCPPClass.h":
cdef cppclass MyCPPClass:
int foo;
string bar;
Sure you have to basically redeclare the .h definition of your class in a Pythonic format, but that's a small price to pay for getting access to your already written C++ functions.
Cython is mainly for C development, to integrate C++ with Python I would recommend Boost.Python. Their excellent documentation should get you started pretty quickly.
The answers above have more or less answered the OP's question.
But its now passing the mid of 2020 and I'd thought to put a contribution (in the form of resource summary) here for those who want to explore Python-C++ bindings via python packages that can be 'pip installed' (or in the PyPI).
I'd reckon for calling C++ from Python / wrapping over third party libs , one can look at cppyy. Its a relatively young project but it certainly looks promising and backed with a modern compiler. See the link below:
https://cppyy.readthedocs.io/en/latest/
Also, there's always pybind11 ....
https://github.com/pybind/pybind11
Cython also supports C++ natively (for most of the C++ languages) ; for more information see https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html