How does ND4J reference native libraries? - java-native-interface

I am currently working on Deeplearning4J for creating neural networks, what continues to perplex me is that how it references native libraries like OpenBLAS for its operations ?
What does it use ? Does it use jniLoader or some other API ? What is now the de facto piece of software that is used nowadays to write JNI programs ?
Also what's more confusing is that the OpenBLAS libraries are different on both Linux and Windows ? Does that mean that the API of the libraries are same ? I don't see how thats possible since native libraries are bound to the platform they are compiled on.

It depends on the backend. We use We use Java's service loader interface that usually delegates to some JNI layer underneath.
Happy to answer more specific questions if you're looking for something in particular.
Hope that makes sense!It depends on the backend. We use We use Java's service loader interface that usually delegates to some JNI layer underneath. Hope that makes sense!

Related

How can I tell if the libaries, dlls, and functions I use are C++ Native

I am learning C++ with the intention to make programs with zero dependencies.
I am noticing that throughout my code I later find out that a libary/dll/functions I have been using requires the machine running it to have .NET installed or some other package.
How can I detect this beforehand?
Are most things that are possible with these .NET libaries/dlls/functions possible with only Native C++ libaries/dlls/functions?
I assumed so.
How can I detect this beforehand?
Basically don't use anything beyond C++ standard library, which is described here or here.
A tool like Dependency Walker can be handy.
You can also use static libraries to make program less dependent on DLLs.
Are most things that are possible with these .NET
libaries/dlls/functions possible with only Native C++
libaries/dlls/functions?
The point is modern software must depend on some underlying framework and libraries, because it's pointless/impossible to write everything from scratch using only standard library. If you want to write rich application you need something like .NET. It can be cross-platform framework tho, like qt, which will give you more flexibility.

Can I use pure native C++ to write apps for windows 8 metro?

With native c++, I mean, not managed c++, not cli, not any special things from microsoft, I can:
1) get high performance
2) use existing c++ code library and engine
3) write cross platform code (for example, for ios and android)
it needn't be fully native c++, I can use managed code to do the ui things, like object-c in ios and java in android, but beside interface, can I use native c++ code?
I suggest you have a look at the presentation here: Using the Windows Runtime from C++ and especially at the comments from Herb Sutter. I quote:
Please answer this question: If I decide to write C++ GUI application
in Metro style am I forced to use all these proprietary ref, sealed,
^, Platform::String^ extensions for GUI components or not?
#Tomas: No, you are not forced to use them. We are providing two
supported ways:
1) These language extensions (C++/CX).
2) A C++ template library (WRL), see
Windows Kits\8.0\Include\winrt\wrl as Yannick mentioned. WRL is a C++
library-based solution sort of along the lines of ATL, which offers
what I think you're looking for -- template wrapper/convenience
classes and explicit smart pointers and such.
Yes you absolutely can, real native C++ is fully supported.
You do however mostly have to use the new WinRT libraries to do an user interface or system calls and although they are native code and fully callable from C++ directly the interface to them makes it very painful indeed to do so, as everything is a reference counted COM object and in addition it's not so easy to create instances of them as just calling "new" so you have to write a lot of ugly code to do so.
As the earlier answer said, microsoft provide two ways to help with this. One is via language extensions to c++ and the other is a c++ template library. Personally I consider both to be rather ugly for doing something as simple as calling an API but that's just me :)
But to answer your question, it's completely possible to write your application in real native c++. You won't need to use managed code at all for anything. But you'll probably want to use either the language extensions or the template library to make calling the API more easy.
Personally I'm hoping someone writes a wrapper for WinRT that exposes the most necessary functionality as a more usable c++ native library and then everyone can just use that from c++ instead...

WCF-like Native C++ Library

I am looking for a library for native C++ (library source code has to be gcc compatible and portable across Linux and Windows) that does what WCF does in its very basic form - i.e. OperationContracts and DataContracts in a client-server environment, with data exchanges in binary format (binary serialization).
Ideally I'd like to use a library to achieve this. So if there's a library already available that compiles OperationContracts and DataContracts into rich C++ classes with metadata for reflection which can be consumed in our code and with client-server TCP communications built-in (i.e. a rudimentary implementation of WCF's functionality without the need to be compatible with WCF at all), please point me to it.
If not, implementing them myself (unlikely due to time constraints), I could use boost::serialization for DataContracts but how would I implement OperationContracts?
It's not necessarily compatible with gcc--so it's somewhat tangential to the precise question asked here--but I'd like to include a reference to the Windows Web Services API, Microsoft's native-code counterpart to WCF (for Windows systems).
From this secondary article:
WWS is designed from the ground up to be a completely native-code implementation of SOAP, including support for many of the WS-* protocols. WWS is, strictly speaking, exposed through a C API, making interoperability with other languages and runtimes very straightforward, but it is the C++ developer who will likely benefit the most.
I have found one that fits the purpose called "RCF" (Remote Call Framework) by Delta V Software. It's open source (GPLv2 or US$195 closed source). So far in my testing, it's working very well. According to the site, companies like HP, Ericsson and Siemens are users of the library.
Apache Thrift is another option you might consider,
http://en.wikipedia.org/wiki/Apache_Thrift

Implementing A Plugin System in C or C++ [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
What are your tips on implementing a plugin style system?
In C (and I think C++ too although I haven't done it myself), this is most typically done using dynamically loaded modules. The API:s for such are platform-dependent.
On POSIX (Linux), you use the dlopen() family of functions. Basically you build your plugin separately, then load it at run-time, look up its symbols by name, and can then call them.
For Win32, there is LoadLibrary() which does something very similar, you build your code into a DLL.
For a handy wrapper that makes all of these easy and transparent, check out GLib's GModule API.
In the '92/'93 time frame I worked on a plugin architecture for Aldus PageMaker, which was coded in C++. PageMaker was built on a C++ OOP framework called VAMP, which assisted its portability between Mac OS and Windows.
So we tried to use the features of C++ to build a plugin architecture. This proved to be very problematic for C++ classes due to the so-called brittle base class problem. I proceeded to write a paper that was published in journals and that I presented at OOPSLA '93 in a reflection workshop. I also made contact with Bjarne Stroustrup at a Usenix conference in Portland and proceeded to dialog with him for several months, where he championed the issue of dealing with the brittle base class problem on my behalf. (Alas, other issues were deemed more important at that time.)
Microsoft introduced the COM/DCOM system and for that platform that was looked on as a viable solution to the problem. C++ could be used as an implementation language for COM via abstract classes used to define COM interfaces.
However, these days developers shun away from COM/DCOM.
In contrast, NeXT devised a plugin architecture using Objective C in the early 90s in the NeXT Step framework. Today that lives on vibrantly in Mac OS X on Apple's computers and important platforms such as the iPhone.
I submit Objective C enabled solving the plugin problem in a superior manner.
I personally regard the brittle base class problem of C++ to be it's most fatal flaw.
If were building a plugin architecture with the C-based family of languages, would do so using Objective C.
The best platform and language neutral advice I can give is this:
Design your entire app around the plugin SDK.
IMO, a plugin SDK should not be an afterthought. If you design your app to basically be an empty shell which loads plugins, then the core features are implemented in your own SDK, you get the following benefits:
High modularity of components, and clear separation of purpose (it kind of forces your architecture to be good)
It forces your SDK to be really good
It allows other third party developers to make extremely powerful, core-level features as well
New developers/hires can easily start work on a major new feature without having to touch the main app - they can do all their work in a plugin (which prevents them screwing up anything else)
In C/C++, you probably use dynamic link libraries and either function pointers (C) or interfaces (classes solely consisting of pure virtual methods, for C++). However even if you use Javascript, I'd still recommend the above architecture.
Qt provides QPluginLoader:
http://qt-project.org/doc/qt-4.8/qpluginloader.html
If you need/want more fine grained control, Qt also provides a means to load libraries on the fly with QLibrary:
http://qt-project.org/doc/qt-4.8/qlibrary.html
Even better, these are portable across platforms.
This may not be what you're looking for, but you could embed a scripting language in your application, such as Lua. Lua was designed to be embedded in other programs and used as a scripting language for writing plugins. I believe it's fairly easy to add the Lua interpreter to your program, though I don't know Lua so I can't vouch for how effective of a solution this would be. Others with more experience with Lua, please add comments about your experience with embedding Lua in another application.
This would, of course, mean that your plugins need to be written in Lua. If you don't like Lua then the de-facto standard Perl, Python and Ruby interpreters are all written in C, and can be embedded in a C program. I know of a number of programs that use these languages as scripting language extensions.
However, I don't know what you're looking for, as your question is a little vague. Perhaps more information about what you want people to be able to do with said plugins would be appropriate. For some tasks, a full-blown scripting language may be a bit overkill.
I have written an article about how to implement a plugin system using Dynamic Linking Libraries. The article is written from the point-of-view of a Windows programmer but the technique can be applied to a Linux/Unix type environment.
The article can be found here: http://3dgep.com/?p=1759
The main point is, you should create a "common" DLL that is implicitly linked by both the main application (the core application) and by the plugin implementations. The plugins can then be explicitly linked and loaded dynamically at run-time by the core application.
The article also shows how you can safely share static (singleton) instance of a class across multiple DLLs by using the "common" DLL.
The article also shows how you can export a "C" function or variables from a DLL and use the exported functions in the application at run-time.
It's best to use a framework like ACE (http://www.cs.wustl.edu/~schmidt/ACE.html) that shields you (as good as possible) from platform specific coding.
ACE contains a plugin framework that is based on shared libraries that you can use to create dynamically assembled applications.
For a higher level abstraction check out CIAO (http://www.cs.wustl.edu/~schmidt/CIAO.html) the Open Source C++ implementation of the CORBA Component Model.
Look at Poco Class Loader, it can be interesting for you.
I have written a plugin library Pugg that loads C++ classes from dll files and here is the logic I used:
User exports a c function from dll that has a unique name. This name has to be unique enough as functions cannot be distinguished using their arguments while loading from dlls.
C function registers one or several factory classes called "Driver". Every Driver class is associated with a string. When the main application wants to create a class, it gathers the related factory class using the associated string. I also implemented a version checking system to not load old plugins.
Dll loading is accomplished using the LoadLibraryA and GetProcAddress functions (Pugg currently works on windows).
One thing worth mentioning is that main application and dlls should be compiled using the same compiler and using the same compilation options (release/debug modes, optimization settings, stl versions etc...). Otherwise there might be issues with mapping of classes.
I have had some success using a fairly naive system:
Create API Specification for plug-ins
Use a singleton plug-in manager
Use LoadLibrary/GetProcAddress based run time dynamic linking
Implement Inversion of control based event handling for notifying the plug-ins
This podcast on plugin architectures might also be interesting.

What kind of code library should I build for distribution?

I need to build a C++ library to distribute among our customers. The library must be able to be accessed from a wide range of languages including VB6, C++, VB.net and C#.
I've being using ActiveX controls (ocx files) until now. But I wonder if there is a better kind of library (dll, etc.) that I can build. What do you recommend?
I'm limited to C++ as the library language, but you can mention other languages for reference to other developers.
P.S. Sorry if the question was already asked. I had some trouble finding a suitable title. Feel free to correct my English.
Edit: Seems like the best choice is either DLLs or OCX (i.e., COM), but I'm still having some doubts on which one will I choose. Which one is more suitable to modern languages (.NET for instance)? Which one would be easier to use from an end developer perspective?
Almost every language has a way of loading dynamic libraries and accessing exported C functions from them.
There is nothing preventing you from using C++ inside the dll but for maximum portability, export only C functions.
I have some more about this in this post.
If you're looking at supporting both VB6 and .NET, you're pretty much stuck with exposing interfaces via COM, but at least that'll get you out of having to create more than one wrapper based on the language/runtime system you're trying to interact with.
If there is any chance this will need to be ported to non windows platforms then a DLL / Shared library is your best choice as a COM object really isn't at all portable.
In addition you can call a DLL from almost any platform even if it requires you to write a wrapper of some kind. It's pretty easy to wrap a dll in a com object but if you make a native com object it's a lot harder to add a C style DLL API. Plus you might want to call it from java for example and it's much easier to write a JNI wrapper to call your DLL than get it working with COM in any kind of cross platform way.
Really it depends on what platforms you really need to call it from and how certain you can be that you won't get something out of the ordinary in future.
To be callable from all those languages your only real option is going to be COM, without having to write wrappers where required (which would defeat the point)