I would like to be able to use C/C++ functions from python using ctypes python module.
I have a function int doit() in the .c / .cpp file. When I try to load the shared library:
Frr=CDLL("/path/FoCpy2/libFrr.so")
Frr.doit(c_int(5))
i find it working really well when the .c variant is used. When C++ is called the good way to call this function is (found out using nm libFrr.so using nm -gC libFrr.so produces just plain doit()):
Frr._Z4doitv(c_int(5))
I have read on Stackexchange that there is no standard way to call C++ from python, and there is "nonstandard name mangling" issue. Is the "Z4" a part of that issue? I guess the nonstandard name mangling would appear for more advanced language features such as class methods, templates, but also for such basic functions? Is it possible to force using simple C function names in simple cases for the C++ code?
You can use extern "C" to make the functions "look like" C functions to the outside world (i.e., disable name-mangling). And yes, you are correct, name-mangling is needed mostly for the more complicated features and types of functions that C++ has, and the name-mangling scheme has never been standardized (nor the binary compatibility) and so it varies from compiler to compiler and between versions (but most main-stream compilers have settled to something permanent now, but still different between compiler-vendors). And the reason that mangling is also required for plain old free functions is because C++ supports overloading (same function names but with different parameters), and thus, the compilers will encode the parameter specification (e.g., types) into the mangled names. Of course, if you use extern "C" you lose all features for which name-mangling is needed, so, it more or less boils down to C functions only.
You can use extern "C" either on a per-function basis, like so:
extern "C" int doit();
Or for the overall header:
extern "C" {
// all the function declarations here ...
};
However, for Python specifically, I highly recommend that you use a library that allows you to construct Python classes and functions that are a reflection of your C++ classes and functions, that makes life a lot easier and hides away all this extern "C" business. I recommend using Boost.Python, see this getting started page, it makes exporting functions and classes to Python a breeze. I guess others would also recommend SWIG, but I have never used it.
Calling c++ library functions is always a mess, actually even if you're using C++ you have to use the same compiler, etc. to make sure it works.
The only general solution is to define your c++ functions as extern "C" and make sure you follow the involved limitations - see here for an explanation of it.
Related
I know I can include C methods inside a C++ project using the extern "C" thing. But lets us now suppose that I'm thinking in creating a C++ project that would use quite a lot of C methods coming from both libraries made by me as well as libraries made by other people/companies whose developing details and compilation specifications I'm simply not aware of.
Is it possible that some of this methods of C libraries, with unknown compilation and configuration details, could not be included in my C++ project with extern "C"? Or are all C methods necessarily 100% compatible with C++ code insofar extern "C" is used?
It's possible that some of the functions exported by C use names that collide with C++ keywords. You wouldn't be able to declare those using extern "C".
Functions exported by assembler could even use names that conflict with C keywords.
Those and functions declared static can still be called via function pointer, as long as the library gives you a way to get one.
Headers might not be parse-able in C++ mode for the same reasons -- things like the restrict keyword.
Other than naming issues, C++ has full support for the C calling convention. That's what extern "C" is all about.
C has constructs for interfaces that are not compatible with C++, in particular variable length arrays. In modern C you would write
void matMult(size_t n, size_t k, size_t m, double A[n][k], double B[k][m], double C[n][m]);
this interface can not be included as such in C++ compilation units.
Although rather unlikely, one possible issue that might arise with extern "C" in-place is when a function pointer declared extern "C" points to a C++ function that is not declared extern "C". See the last part of this page for more details.
I have compiled a library that I have created with MinGW into an existing application using Borland 6 (I know its old but that's what it was made with). I have used implib to create the .lib file and imported it into my project. I have also added the paths to the dll and necessary header files.
When I try to compile I get a pile of Unnresolved external type errors. Have I missed out any steps of the importing process? Assuming I haven't and the issue is something like name-mangling how do I go about writing the interface in such a way that name mangling won't matter. I know it involves extern C but thats about the limit of my knowledge. There are only two classes that need to be accessed from outside the dll the remainder are all only used internally. I'm not sure how to use extern C with something that is entirely built with classes. I'm stil hopeful that it's my importing with borland 6.
extern "C" cannot be used for classes, only for free functions. So you have an option of writing a "C" interface to your class, where each function takes a pointer to your class and you would probably have create and destroy functions.
This is a way it is typically done, and your class could be forwardly declared as struct, which is the same as class, and then could even be used by applications written in C. You would normally put extern "C" only when __cplusplus is defined so there are normally #ifdef guards around it.
There is another option, if you only want your class to be used by C++ and you don't want to have to write a C interface for all your class methods.
Users of the DLL use an abstract interface and still use Create and Destroy methods (with extern "C") to create a pointer to the abstract interface, but then use the pointer in the normal C++ way. Of course ideally you will wrap this pointer in a smart pointer, e.g. a boost shared_ptr with a custom deleter that calls the Destroy method. (Users of the library should do that but you can provide a headers-only interface to do it).
There are a few other issues you would need to beware of if doing this, e.g. anything to do with run-time type information is likely to not work on the user-side, including exceptions. And once again your library could provide "open-source" C++ wrappers (compiled on the client side) to handle this in a more C++ way. A sort-of pImpl.
The name mangling is not standardized across compilers. Only expose extern C functions so that they are not name mangled. But this has a limitation that you cannot use object orient programming.
Another option is to implement COM objects as they are binary compatible.
I'm trying to create a abstraction layer for platforms such as win32, mac os, linux, iOs etc.
I want this to be dynamically linked. On platforms that don't support this it shouldn't be a problem since from what I've seen everything that can be compiled as a dynamic library can be compiled as a static one as well with minimum impact on the code.
Now, to get to the point of this:
I created a interface named IThread and a class named CThread. I use a function named CreateThread that's defined with extern "C" to be able to export it and call it outside the library. The problem here is that in win32 for example there already is a function named CreateThread and thus I get a linker error. I understand the error and why it's appearing but I'm not sure of a good way to avoid this. I don't really like to use weird naming as qt uses like CreateQtThread.
Another idea I have would be is to create a thread manager/factory that creates instances of CThread but I'm not sure this would be a great idea.
What do you guys think about this? I'm asking because I don't want to rush on important organizing problems like this.
Thank you very much
I use a function named CreateThread that's defined with extern "C" to be able to export it and call it outside the library.
This is bad. I can't talk for other platforms, but on windows it's perfectly fine to export C++ functions. They are just get mangled, and you get some sanity checking in case someone changes the declaration. In fact, it's the only correct way to export a function that is C++. If you declare it as extern "C" you get no namespaces, no overloading, and someone who compiles with /EHsc will be in trouble if an exception escapes from your function.
Preferred solution: Do not declare them as extern "C", and put them in a namespace.
The only other solution: well, guess why all those C libraries prefix their functions with their_lib_prefix_function...
Your decision to use extern "C" is sound in my view because it allows access from other languages, compilers etc. But doing so means you can't use namespaces so you simply must prefix your functions with something to identify them as being from your library, a poor man's namespace if you will.
This is the compromise you must make if you want to use extern "C".
Well, I don't know whether you will like this as I know the C developers I worked with found it unaesthetic. But, it's very simple and prevents collisions like this. (More or less mentioned in this answer with the "their_lib_prefix_function" comment.)
So, whenever I was using C code, I used to basically 'namespace' my functions using an underscore. So, let's say your namespace is MegaAbstraction, then something like CreateThread becomes MegaAbstraction_CreateThread. Easy peasy. And no collisions, unless someone else has a namespace called MegaAbstraction in which case I recommend finding a namespace that's unlikely to be used by anyone else. ;)
Does your CreateThread use the stdcall calling convention (aka WINAPI)? If you use the cdecl calling convention, it should export the function name as _CreateThread and avoid the linkage conflict with the Win32 CreateThread function.
extern "C" _declspec(export) int _cdecl CreateThread(...
There is no "CreateQtThread" function in Qt, not even something similar. There's a QThread class, and it has constructors. If needed, you can put everything in a namespace.
The Python manual says that you can create modules for Python in both C and C++. Can you take advantage of things like classes and templates when using C++? Wouldn't it create incompatibilities with the rest of the libraries and with the interpreter?
It doesn't matter whether your implementation of the hook functions is implemented in C or in C++. In fact, I've already seen some Python extensions which make active use of C++ templates and even the Boost library. No problem. :-)
The boost folks have a nice automated way to do the wrapping of C++ code for use by python.
It is called: Boost.Python
It deals with some of the constructs of C++ better than SWIG, particularly template metaprogramming.
What you're interested in is a program called SWIG. It will generate Python wrappers and interfaces for C++ code. I use it with templates, inheritance, namespaces, etc. and it works well.
You should be able to use all of the features of the C++ language. The Extending Python Documentation (2.6.2) says that you may use C++, but mentions the followings caveats:
It is possible to write extension
modules in C++. Some restrictions
apply. If the main program (the Python
interpreter) is compiled and linked by
the C compiler, global or static
objects with constructors cannot be
used. This is not a problem if the
main program is linked by the C++
compiler. Functions that will be
called by the Python interpreter (in
particular, module initialization
functions) have to be declared using
extern "C". It is unnecessary to
enclose the Python header files in
extern "C" {...} — they use this form
already if the symbol __cplusplus is
defined (all recent C++ compilers
define this symbol).
The first restriction, "global or static objects with constructors cannot be used", has to do with the way most C++ compiler initialize objects with this type of storage duration. For example, consider the following code:
class Foo { Foo() { } };
static Foo f;
int main(int argc, char** argv) {}
The compiler has to emit special code so that the 'Foo' constructor gets invoked for 'f' before main gets executed. If you have objects with static storage duration in your Python extension and the Python interpreter is not compiled and linked for C++, then this special initialization code will not be created.
The second restriction, "Functions that will be called by the Python interpreter (in particular, module initialization functions) have to be declared using extern "C"", has to do with C++ name mangling. Most C++ compilers mangle their names so that they can use the same linkers provided for C toolchains. For example say you had:
void a_function_python_calls(void* foo);
the C++ compiler may convert references to the name 'a_function_python_calls' to something like 'a_function_python_calls#1vga'. In which case you may get an unresolved external when trying to link with the Python library.
I have a C++ library that provides various classes for managing data. I have the source code for the library.
I want to extend the C++ API to support C function calls so that the library can be used with C code and C++ code at the same time.
I'm using GNU tool chain (gcc, glibc, etc), so language and architecture support are not an issue.
Are there any reasons why this is technically not possible?
Are there any gotcha's that I need to watch out for?
Are there resources, example code and/or documentation available regarding this?
Some other things that I have found out:
Use the following to wrap your C++ headers that need to be used by C code.
#ifdef __cplusplus
extern "C" {
#endif
//
// Code goes here ...
//
#ifdef __cplusplus
} // extern "C"
#endif
Keep "real" C++ interfaces in separate header files that are not included by C. Think PIMPL principle here. Using #ifndef __cplusplus #error stuff helps here to detect any craziness.
Careful of C++ identifiers as names in C code
Enums varying in size between C and C++ compilers. Probably not an issue if you're using GNU tool chain, but still, be careful.
For structs follow the following form so that C does not get confused.
typedef struct X { ... } X
Then use pointers for passing around C++ objects, they just have to be declared in C as struct X where X is the C++ object.
All of this is courtesy of a friend who's a wizard at C++.
Yes, this is certainly possible. You will need to write an interface layer in C++ that declares functions with extern "C":
extern "C" int foo(char *bar)
{
return realFoo(std::string(bar));
}
Then, you will call foo() from your C module, which will pass the call on to the realFoo() function which is implemented in C++.
If you need to expose a full C++ class with data members and methods, then you may need to do more work than this simple function example.
C++ FAQ Lite: "How to mix C and C++ code".
Some gotchas are described in answers to these questions:
[32.8] How can I pass an object of a C++ class to/from a C function?
[32.9] Can my C function directly access data in an object of a C++ class?
Main gotcha: exceptions can not be caught in C. If there is the possibility of an exception rising in the C++ code, either write your C code or your C++ wrappers very carefully. Conversely, exception like mechanisms (i.e., longjump) in the C code (as found in various scripting languages) are not required to invoke destructors for C++ objects on the stack.
you can mix C/C++ code. If your main() function in in C++, then you just need to make sure your c functions are declared
extern "C"
If your main is C, then you are probably OK except for static variables. Any constructors with your static variables are supposed to be called before main() start. This won't happen if C is your main. I you have a lot of static variables, the best thing to do is to replace static variables with singletons.