How to load and use native c code in a lein project? - clojure

ProblemI'm unable to load and call methods in a compiled c class into a leiningen project. My basic approach is to load a Java class, JavaWrapper.java, that uses JNI to call some native methods in the native code, wrapper.o and then call the methods through this java wrapper class.
I imagine there are classLoader issues with loading a java class which loads the native code from a clojure project, but given that I can't seem to directly get clojure code to find the wrapper.o on the library path, I'm not sure how to handle this.
lein project file
(defproject lein-native-test "0.1.0-SNAPSHOT"
...
:java-source-paths ["java-path"]
:jvm-opts ["-Djava.library.path=.:./native:/absolute/path/to/native"] ;;not sure what format it wants
)
clojure file with main method
I've tried it slightly modified with four approaches, all included in code below along with respective error in comments.
(ns lein-native-test.core
(:import (com.test JavaWrapper)))
(def -main []
;;four things I've tried and their errors
(clojure.lang.RT.load "/abs/path/to/wrapper.o") ;;could not find file /abs/path/wrapper.o_init.class or wrapper.o.clj
(clojure.lang.RT.loadLibrary "wrapper.o") ;;UnsatisfiedLinkError no wrapper.o in java library path
(JavaWrapper/load "/abs/path/to/wrapper.o") ;;UnsatisfiedLinkError com.test.JavaWrapper.setup()
(assembly-load "/abs/path/to/wrapper.o") ;;unable to resolvesymbol: assembly-load
)
Java code with native methods that uses JNI, JavaWrapper.java
public class JavaWrapper{
public native void setup();
public static void load(String lib){ System.load(lib);}
}
Before trying to get this to work with clojure and lein I did successfully load and use the native methods in wrapper.o via JavaWrapper and JNI.
Possibly related: I'm also unable to load wrapper.o in JavaWrapper.java via
System.loadLibrary("wrapper.o");
I have to use
System.load("/absolute/path/to/wrapper.o");
Versions of tools
clojure version: 1.5.1
lein version: 2.3.4
jdk: 1.7
os: debian7
A better understanding of ClassLoaders or especially a working simple example would be very useful, thanks.

The problem was due to an naming error in my method in the C header and source files per the jni standard. The correct way to use jni with clojure is to create a Java wrapper class as I have done and load the dynamic library with the method
clojure.lang.RT.loadLibrarySince I've had trouble finding good examples for this I've made a demo on github
Errors
1) (clojure.lang.RT.load "/abs/path/to/wrapper.o") ;;could not find file /abs/path/wrapper.o_init.class or wrapper.o.clj
This load method is not meant to be used for native code, its expecting a java class or a clj file
2) (clojure.lang.RT.loadLibrary "wrapper.o") ;;UnsatisfiedLinkError no wrapper.o in java library path
Clojure is unable to find the library at link time, hence UnsatisfiedLinkError --- this is due to a naming error
first the library should be compiled as a dynamic shared library, i.e. for the gcc compiler use the -shared flag (I actually did but didn't name the output file with the normal .so extension)
java and thus clojure expect the native library to be named in a very specific way: libwrapper.so (or .jnilib for mac or .dll for windows but always with "lib" prefix)
3) (JavaWrapper/load "/abs/path/to/wrapper.o") ;;UnsatisfiedLinkError com.test.JavaWrapper.setup()
This time the error is on a method within JavaWrapper in the file or library, thus you know that at least it found the file. An UnsatisfiedLinkError that specifies a specific method in the Java class, like this one, should always be due to a naming error between what is declared a native method in the Java file and what is actually present in the c source or header file.
Note the namespace "com.test"
When declaring a jni method in c the method name must follow a specific format,
from http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/design.html
"Dynamic linkers resolve entries based on their names. A native method name is concatenated from the following components:"
the prefix Java_
a mangled fully-qualified class name
an underscore (_) separator
a mangled method name
for overloaded native methods, two underscores (__) followed by the mangled argument signature
In this case the full c source method signature would be
void Java_com_test_setup(JNIEnv *env, jobject obj)
4) (assembly-load "/abs/path/to/wrapper.o") ;;unable to resolvesymbol: assembly-load
This method too is not meant to load native code

Related

What do I need to do to get cocoapods to build with C++?

I'm trying to build a library to distribute via Cocoapods. The library is written mainly in Objective C, but includes a few C++ files. None of the C++ headers are part of the library's public API. The library builds fine in Xcode, and I can distribute as a framework, but a pod is probably easier for others to consume, right?
This is my first time attempting to build a cocoapod, so I may be doing something obviously wrong.
When I run pod lib lint or try to build the demo app that depends on the pod, I get error messages that suggest that the build system doesn't understand C++ at all.
in a C++ header file:
class CGuard {
> unknown type name 'class'; did you mean 'Class'?
in another C++ header file:
template<typename T>
class CContexts {
> unknown type name 'template'
My podspec file includes
spec.xcconfig = {
'CLANG_CXX_LANGUAGE_STANDARD' => 'c++11',
'CLANG_CXX_LIBRARY' => 'libc++'
}
spec.library = "c++"
What else can I do to tell the build system to use the C++ compiler?
It looks like the key was adding private_header_files entries to remove the C++ headers from the framework that Cocoapods exports.
Although Cocoapods is documented to only include the files from public_header_files in the generated module umbrella header, it looks like each subspec section has all of its headers included if that subspec section doesn't have its own public_header_files entry.
The C++ files were all compiled in a subspec section to let me use different compiler settings, so the private_header_files entry needed to be inside that subspec section as well.

Matlab DLL Integration via C++ or VB.NET

I haven't found a solution to my problem, because of limited knowledge about dll Libraries.
I am using Matlab (R2017a) to integrate communication to a device through a dll, which is provided by the manufacturer (files - Xemo-DLL (64Bit) mit Header-Dateien (2.40)).
As far as I understand there is precompiled .dll in C and a header File in C++, beside a VB and VB.NET wrapper.
Question is: What is the best way to integrate the dll into Matlab?
A) I tried loadlibrary(XemoDll). It throws a lot of errors, iostream wasn't found. So I added extern "C" {} to the whole file, which created new errors at every function definition. Where do I have to insert it?
B) I tried to add NET.addAssembly(path\XemoDll.vb) of the VB.NET library. There is a Module with all wrapper functions. Matlabs gives the error "assembly manifest missing" Source: mscorlib.
I found a Tutorial on loading Dlls with the LoadLibrary() command from windows.h, and got it working.
I use a a DllInterface_mex function that gets the function name and calls the approriate function in the Dll.

How to use third part dll from Haxe

I'm starting project, and I will have to use external dll written on pure C. How to load external dll (NOT.Net libtaty, if it is important) to use it from Neko or Cpp target in Haxe?
I found out the answer on my question. It is trivial, no nedded special settings for the compiller, no special list of dynamic loaded libraries at compile time.
Just load library at runtime using haxe cpp API like below:
static var sum:Int->Int->Int = cpp.Lib.load("test","sum",2);
or haxe neko API according to your target platform:
static var sum:Int->Int->Int = neko.Lib.load("test","sum",2);
This lines load sum function from test library which located in the same directory with executable file.
Read more about it in old haxe documentation.
[1]: http://old.haxe.org/doc/cpp/ffi C Foreign Function Interface

Ruby FFI not finding certain functions?

I'm trying to use Ruby's FFI library to link functions from the bitcoin-core secp256k1 library.
To make the secp256k1_ecdsa_sign function accessible, I built libsecp256k1 using autotools (as directed in README.md). Then I created a shared object to use in FFI by running g++ -shared secp256k1/src/.libs/libsecp256k1_la-secp256k1.o. Importing this into my Ruby file using FFI let me use the function and everything worked perfectly.
I'm trying to do the exact same with the secp256k1_ecdsa_sign_recoverable function, which is in the same C project, just a different header file. However, I get the error Function 'secp256k1_ecdsa_sign_recoverable' not found in [bin/secp256k1.so] (FFI::NotFoundError).
I think this is either because I'm not creating the shared object properly (created with the aforementioned g++ command), or because this function is for some reason not a public-facing one in the C project (though I don't know enough about C to know how to figure out if this is the case).
If someone could help me figure out how to use this function it would be greatly appreciated.
It ends up the secp256k1_recovery.h file is only included if you specify that when building the libsecp256k1 library. Specifically, I needed to run ./configure --enable-module-recovery instead of ./configure.

Is it possible to split a SWIG module for compilation, but rejoin it when linking?

I hit this issue about two years ago when I first implemented our SWIG bindings. As soon as we exposed a large amount of code we got to the point where SWIG would output C++ files so large the compiler could not handle them. The only way I could get around the issue was to split up the interfaces into multiple modules and to compile them separately.
This has several downsides:
• Each module must know about dependencies in other modules. I have a script to generate the interface files which handles this side of things, but it adds extra complexity.
• Each additional module increases the time that the dynamic linker requires to load in the code. I have added an init.py file that imports all the submodules, so that the fact that the code is split up is transparent to the user, but what is always visible is the long load times.
I'm currently reviewing our build scripts / build process and I wanted to see if I could find a solution to this issue that was better than what I have now. Ideally, I'd have one shared library containing all the wrapper code.
Does anyone know how I can acheive this with SWIG? I've seen some custom code written in Ruby for a specific project, where the output is post-processed to make this possible, but when I looked at the feasibility for Python wrappers it does not look so easy.
I just did equivalent hack for TCL library: I use several SWIG modules, generating several .cpp files that are compiled in several .o files but compile them all in a single .so file that is loaded by a single TCL "load" command.
The idea is to creates a top swig module (Top) that calls initialization functions of all sub-modules (Sub1 and Sub2):
%module Top
%header %{
extern "C" {
SWIGEXPORT int Sub1_Init(Tcl_Interp *);
SWIGEXPORT int Sub2_Init(Tcl_Interp *);
}
%}
%init %{
if (Sub1_Init(interp) != TCL_OK) {return TCL_ERROR;}
if (Sub2_Init(interp) != TCL_OK) {return TCL_ERROR;}
%}
There's nothing special in the submodules files.
I end up with file Top.so that I load from TCL with command "load ./Top.so"
I don't know python but's likely to be similar. You may need to understand how the python extensions are loaded, though.
If split properly, the modules don't necessarily need to have the same dependencies as the others - just what's necessary to do compilation. If you break things up appropriately, you can have libraries without cyclic dependencies. The issue with using multiple libraries is that by default, SWIG declares its runtime code statically, and as a result, as problems passing objects from one module to another. You need to enable a shared version of the SWIG runtime code.
From the documentation (SWIG web page documentation link is broken):
The runtime functions are private to
each SWIG-generated module. That is,
the runtime functions are declared
with "static" linkage and are visible
only to the wrapper functions defined
in that module. The only problem with
this approach is that when more than
one SWIG module is used in the same
application, those modules often need
to share type information. This is
especially true for C++ programs where
SWIG must collect and share
information about inheritance
relationships that cross module
boundaries.
Check out that section in your downloaded documentation (section 16.2 The SWIG runtime code), and it'll give you details on how to enable this so that objects can be properly handled when passed from one module to the other.
FWIW, I've not worked with Python SWIG, but have done Tcl SWIG.