I want to call a C++ dynamic library (*.so) from Rust, but I don't want to build it from Rust. Like this,
cc::Build::new()
.file("src/foo.cc")
.shared_flag(true)
.compile("libfoo.so");
In some cases, I only need to call several functions, not all the functions. How can I use it?
Before you go further, make sure you have a basic idea of Rust FFI (foreign function interface).
In Rust, it's easy to call C, but hard to call C++.
To call C functions in Rust, you just have to wrap them with extern, do some basic type casting and sometimes unsafe.
To call C++ functions, since Rust does not have built-in knowledge of C++ features, you may have to do a lot of manual translation. For example, here is part of the documentation from Rust-Qt:
Many things are directly translated from C++ to Rust:
Primitive types are mapped to Rust's primitive types (like bool) and types provided by libc crate (like libc::c_int).
Fixed-size numeric types (e.g int8_t or qint8) are mapped to Rust's fixed size types (e.g. i8).
Pointers, references and values are mapped to Rust's respective types.
C++ namespaces are mapped to Rust submodules.
C++ classes and structs are mapped to Rust structs. This also applies to all instantiations of template classes encountered in the
library's API, including template classes of dependencies.
Free functions are mapped to free functions.
Class methods are mapped to structs' implementations.
Destructors are mapped to Drop and CppDeletable implementations.
Function pointer types are mapped to Rust's equivalent representation. Function pointers with references or class values are
not supported.
static_cast and dynamic_cast are available in Rust through corresponding traits.
Names of Rust identifiers are modified according to Rust's naming
conventions.
When direct translation is not possible:
Contents of each include file of the C++ library are placed into a separate submodule.
Method overloading is emulated with wrapping arguments in a tuple and creating a trait describing tuples acceptable by each method.
Methods with default arguments are treated in the same way.
Single inheritance is translated to Deref and DerefMut implementation, allowing to call base class methods on derived
objects. When deref coercions are not enough, static_cast should be
used to convert from derived to base class.
Getter and setter methods are created for each public class field.
Not implemented yet but planned:
Translate C++ typedefs to Rust type aliases.
Implement operator traits for structs based on C++ operator methods (issue). Operators
are currently exposed as regular functions with op_ prefix.
Implement Debug and Display traits for structs if applicable methods exist on C++ side.
Implement iterator traits for collections.
Subclassing API (issue).
Provide access to a class's public variables (issue).
Provide conversion from enums to int and back (used in Qt API).
Support C++ types nested into template types, like Class1<T>::Class2.
Not planned to support:
Advanced template usage, like types with integer template arguments.
Template partial specializations.
Template methods and functions.
My suggestion is to wrap your C++ library as a C library, then call it the official FFI way, or use rust-bindgen to automatically do the wrapping.
If you still want to call C++ in Rust, rustcxx seems like a handy tool.
As to the library linking, it's pretty simple:
Put the library into your system library searching paths like /usr/lib or /usr/local/lib/, make sure it can be found by ldconfig -p.
Or use the environment variable LD_LIBRARY_PATH to specify the path where your library lays when you run cargo from the CLI.
According to Rust's official website, there is no official support for linkage with C++. Instead you can try and use C libraries.
There is also a thread on this issue in their users forum, where users suggest some 3rd party projects aiming to tackle this issue:
bindgen - Auto generation of FFI for Rust
cpp-to-rust - Allows to use C++ libraries from Rust. The main target of this project is Qt.
I didn't use them, so I can't recommend anything particular or share my experience, but good luck :)
I feel it's worth mentioning as of now there is CXX library in crate
https://docs.rs/cxx/latest/cxx/
https://cxx.rs/
This is what I will be using going forward in order to interoperate with a C++ Library from a vendor.
The assigned answer has a link but it's publicly archived and all active work is likely here :
https://github.com/dtolnay/cxx
Related
I want to call a C++ dynamic library (*.so) from Rust, but I don't want to build it from Rust. Like this,
cc::Build::new()
.file("src/foo.cc")
.shared_flag(true)
.compile("libfoo.so");
In some cases, I only need to call several functions, not all the functions. How can I use it?
Before you go further, make sure you have a basic idea of Rust FFI (foreign function interface).
In Rust, it's easy to call C, but hard to call C++.
To call C functions in Rust, you just have to wrap them with extern, do some basic type casting and sometimes unsafe.
To call C++ functions, since Rust does not have built-in knowledge of C++ features, you may have to do a lot of manual translation. For example, here is part of the documentation from Rust-Qt:
Many things are directly translated from C++ to Rust:
Primitive types are mapped to Rust's primitive types (like bool) and types provided by libc crate (like libc::c_int).
Fixed-size numeric types (e.g int8_t or qint8) are mapped to Rust's fixed size types (e.g. i8).
Pointers, references and values are mapped to Rust's respective types.
C++ namespaces are mapped to Rust submodules.
C++ classes and structs are mapped to Rust structs. This also applies to all instantiations of template classes encountered in the
library's API, including template classes of dependencies.
Free functions are mapped to free functions.
Class methods are mapped to structs' implementations.
Destructors are mapped to Drop and CppDeletable implementations.
Function pointer types are mapped to Rust's equivalent representation. Function pointers with references or class values are
not supported.
static_cast and dynamic_cast are available in Rust through corresponding traits.
Names of Rust identifiers are modified according to Rust's naming
conventions.
When direct translation is not possible:
Contents of each include file of the C++ library are placed into a separate submodule.
Method overloading is emulated with wrapping arguments in a tuple and creating a trait describing tuples acceptable by each method.
Methods with default arguments are treated in the same way.
Single inheritance is translated to Deref and DerefMut implementation, allowing to call base class methods on derived
objects. When deref coercions are not enough, static_cast should be
used to convert from derived to base class.
Getter and setter methods are created for each public class field.
Not implemented yet but planned:
Translate C++ typedefs to Rust type aliases.
Implement operator traits for structs based on C++ operator methods (issue). Operators
are currently exposed as regular functions with op_ prefix.
Implement Debug and Display traits for structs if applicable methods exist on C++ side.
Implement iterator traits for collections.
Subclassing API (issue).
Provide access to a class's public variables (issue).
Provide conversion from enums to int and back (used in Qt API).
Support C++ types nested into template types, like Class1<T>::Class2.
Not planned to support:
Advanced template usage, like types with integer template arguments.
Template partial specializations.
Template methods and functions.
My suggestion is to wrap your C++ library as a C library, then call it the official FFI way, or use rust-bindgen to automatically do the wrapping.
If you still want to call C++ in Rust, rustcxx seems like a handy tool.
As to the library linking, it's pretty simple:
Put the library into your system library searching paths like /usr/lib or /usr/local/lib/, make sure it can be found by ldconfig -p.
Or use the environment variable LD_LIBRARY_PATH to specify the path where your library lays when you run cargo from the CLI.
According to Rust's official website, there is no official support for linkage with C++. Instead you can try and use C libraries.
There is also a thread on this issue in their users forum, where users suggest some 3rd party projects aiming to tackle this issue:
bindgen - Auto generation of FFI for Rust
cpp-to-rust - Allows to use C++ libraries from Rust. The main target of this project is Qt.
I didn't use them, so I can't recommend anything particular or share my experience, but good luck :)
I feel it's worth mentioning as of now there is CXX library in crate
https://docs.rs/cxx/latest/cxx/
https://cxx.rs/
This is what I will be using going forward in order to interoperate with a C++ Library from a vendor.
The assigned answer has a link but it's publicly archived and all active work is likely here :
https://github.com/dtolnay/cxx
Go vs C++ interoperability has been discussed many times so far (here, for example), and generally it has been shown that it's possible to wrap C++ functions in C functions, and than to wrap C functions in CGO functions. This is the way how to use C++ code in Go application. However, the most of examples consider only functions with parameters of primitive data types (like int or char *).
But what if C++ function expects STL containers like std::vector or std::set to be passed? There are no equivalents for them in C, so I have no idea what to pass from C to C++.
static Status GetApproximateMemoryUsageByType(
const std::vector<DB*>& dbs,
const std::unordered_set<const Cache*> cache_set,
std::map<MemoryUtil::UsageType, uint64_t>* usage_by_type);
UPD
Just for the reference, here is the example of how this problem was solved. This is a real function from RocksDB C++ API. In order to use it in Go, it should be wrapped in function (or several functions) that are written in C++, but look like as just they're written in plain C. These wrappers are compiled to library with Go binary links against.
I'm being curious about if it is possible to override an implemented function. I mean, is there any legal syntax of function declaration / implementation that allows alternative implementation?
Why am I asking? (I know it sounds ridiculus)
First, just of curiosity and expanding my knowledge.
Second, I've learned that the global new can be overrided (Although it is strongly not recommended).
Third, assume that I have written a library: AwsomeLibrary.hpp, which
my friend wants to include.Among a lot of functions, there is a function like void sort(int* arr), which he thinks that he could implement better (and of course call it with the same name).
I mean, is there any legal syntax of function declaration /
implementation that allows alternative implementation?
No. That would break the one-definition rule (ODR).
Second, I've learned that the global new can be overrided (Although
it is strongly not recommended).
Replaceable allocation functions as documented at http://en.cppreference.com/w/cpp/memory/new/operator_new are really just a very special case, a grey area between language and standard library; certainly not something from which you can infer general rules for your own code.
Third, assume that I have written a library: AwsomeLibrary.hpp, which
my friend wants to include. Among a lot of functions, there is a
function like void sort(int* arr), which he thinks that he could
implement better (and of course call it with the same name).
Such problems are beyond the scope of C++. They are more related to source control versioning systems like Git. If, for example, your project is under Git control, then your friend could create a branch of the code with his better implementation.
It is not possible at language level, aside from one "bizarre" language feature you mentioned yourself: replaceable operator new and operator delete functions. These functions can be replaced through a dedicated mechanism, which is why it is formally referred to as replacement (as opposed to overriding or overloading). This feature is not available to the language user for their own functions.
Outside the limits of standard language you can employ such implementation-specific features as weak symbols, which would allow you to create replaceable functions. For example, virtually all functions in GNU standard C library are declared as weak symbols and can be replaced with user-provided implementations.
The latter is exactly what would facilitate replacement of void sort(int* arr) function in your library. However this does not look like a good design for a library. Function replacement capability should probably reserved for debugging/logging and for other internal library-tuning purposes.
When designing a C++ library, I read it is bad practice to include standard library containers like std::vector in the public interface (see e.g. Implications of using std::vector in a dll exported function).
What if I want to expose a function that takes or returns a list of objects? I could use a simple array, but then I would have to add a count parameter, which makes the interface more cumbersome and less safe. Also it wouldn't help much if I wanted to use a map, for example. I guess libraries like Qt define their own containers which are safe to export, but I'd rather not add Qt as a dependency, and I don't want to roll my own containers.
What's the best practice to deal with containers in the library interface? Is there maybe a tiny container implementation (preferably just one or two files I can drop in, with a permissive license) that I can use as "glue"? Or is there even a way to make std::vector etc. safe across .DLL/.so boundaries and with different compilers?
You can implement a template function. This has two advantages:
It lets your users decide what sorts of containers they want to use with your interface.
It frees you from having to worry about ABI compatibility, because there is no code in your library, it will be instantiated when the user invokes the function.
For example, put this in your header file:
template <typename Iterator>
void foo(Iterator begin, Iterator end)
{
for (Iterator it = begin; it != end; ++it)
bar(*it); // a function in your library, whose ABI doesn't depend on any container
}
Then your users can invoke foo with any container type, even ones they invented that you don't know about.
One downside is that you'll need to expose the implementation code, at least for foo.
Edit: you also said you might want to return a container. Consider alternatives like a callback function, as in the gold old days in C:
typedef bool(*Callback)(int value, void* userData);
void getElements(Callback cb, void* userData) // implementation in .cpp file, not header
{
for (int value : internalContainer)
if (!cb(value, userData))
break;
}
That's a pretty old school "C" way, but it gives you a stable interface and is pretty usable by basically any caller (even actual C code with minor changes). The two quirks are the void* userData to let the user jam some context in there (say if they want to invoke a member function) and the bool return type to let the callback tell you to stop. You can make the callback a lot fancier with std::function or whatever, but that might defeat some of your other goals.
Actually this is not only true for STL containers but applies to pretty much any C++ type (in particular also all other standard library types).
Since the ABI is not standardized you can run into all kinds of trouble. Usually you have to provide separate binaries for each supported compiler version to make it work. The only way to get a truly portable DLL is to stick with a plain C interface. This usually leads to something like COM, since you have to ensure that all allocations and matching deallocations happen in the same module and that no details of the actual object layout are exposed to the user.
TL;DR There is no issue if you distribute either the source code or compiled binaries for the various supported sets of (ABI + Standard Library implementation).
In general, the latter is seen as cumbersome (with reasons), thus the guideline.
I trust hand-waving guidelines about as far as I can throw them... and I encourage you to do the same.
This guidelines originates from an issue with ABI compatibility: the ABI is a complex set of specifications that defines the exact interface of a compiled library. It is includes notably:
the memory layout of structures
the name mangling of functions
the calling conventions of functions
the handling of exception, runtime type information, ...
...
For more details, check for example the Itanium ABI. Contrary to C which has a very simple ABI, C++ has a much more complicated surface area... and therefore many different ABIs were created for it.
On top of ABI compatibility, there is also an issue with Standard Library Implementation. Most compilers come with their own implementation of the Standard Library, and these implementations are incompatible with each others (they do not, for example, represent a std::vector the same way, even though all implement the same interface and guarantees).
As a result, a compiled binary (executable or library) may only be mixed and matched with another compiled binary if both were compiled against the same ABI and with compatible versions of a Standard Library implementation.
Cheers: no issue if you distribute source code and let the client compile.
If you are using C++11, you can use cppcomponents. https://github.com/jbandela/cppcomponents
This will allow you to use among other things std::vector as a parameter or return value across Dll/or .so files created using different compilers or standard libraries. Take a look at my answer to a similar question for an example Passing reference to STL vector over dll boundary
Note for the example, you need to add a CPPCOMPONENTS_REGISTER(ImplementFiles) after the CPPCOMPONENTS_DEFINE_FACTORY() statement
I've begun running into slight issues with certain functions that I've bound to lua using luabind. These functions return or take std:string objects as arguments, and the data that c++ eventually gets is garbage, apparently because lua doesn't recognize the datatype. I'm fine with manually binding std:: classes over as I need them, but I have to wonder if that's already been done for the c standard library anywhere. I can't seem to find any sort of bindings library; does anyone know if such a project exists?
Luabind already has automatic conversions between Lua strings and std::string (as well as char*). So you should be able to use them as parameters and/or return values just fine. If you are unable to do so, then something is going wrong, either on your side or on Luabind's side.
Luabind also has an adapter to convert a return type that conforms to an STL-style container into a Lua iterator function. Thus, you can use it in a for-loop like this:
for object in CppFuncThatReturnsStdVector() do
--Do something with "object"
end
Other than these, I am not aware of any particular effort to make a Luabind library that directly exposes the standard C++ library to Lua. It probably wouldn't be a good idea anyway; Lua tables cover most of the needs you might have for STL-style containers. And most of the other stuff are things Lua can handle with its own standard library.