I have a task to interface with a dll from a third party company using C++.
The dll package comes with:
the dll itself
a sample Java implementation - consists of a java wrapper(library) generated using the SWIG tool and the the java source file
documentation that states all the public datatypes, enumeration and member functions.
My other colleague is using Java(based on the example in package) to interface with the dll while I'm asked to use C++. The Java example looks straight forward... just import the wrapper and instantiate any class described in the docs..
More info on the dll:
From the docs, it says the dll was programmed using C++
From a hexdump, it shows that it was compiled using VC90 (VS C++ 2008 right?) and something from Dinkumware.
From a depends.exe output, the functions seems to be wrapped under JNI. For example: _Java_mas_com_oa_rollings_as_apiJNI_Server_1disconnect#20
My dilemma:
The dll company is not changing anything in the dll and not providing any other info.
How do i use the member functions in the class from the dll?
I did some simple LoadLibrary() and GetProcAddress and manage to get the address of the public member functions.
But i dunno how to use the functions that has the datatype parameters defined in the dll. For example:
From the docs, the member function is defined as:
void Server::connect(const StringArray, const KeyValueMap) throw(std::invalid_argument,std::out_of_range)
typedef std::map Server::KeyValueMap
typedef std::vector Server::StringArray
how do i call that function in C++. The std::map and std::vector in my compiler (VS 2005) has different functions listing that the one in the dll. For example, from the depends.exe output:
std::map // KeyValueMap - del, empty, get, has_1key,set
std::vector // StringArray - add, capacity, clear, get, isEMPTY, reserve, set, size
Any advice/strategy on how i should solve this? Is it possible to simply instantiate the class like the Java example?
If you are trying to use VS 2005 to try and interface with a DLL that is built using VS2008, your attempts will be mostly doomed unless you can use a plain C interface. Given your description, this is not the case; The runtime libraries differ between VS2005 and VS2008 so there is little chance that the object layout has stayed the same between compilers. The 'something from Dinkumware' that you're referring to is most likely the C++ standard library as ISTR that Microsoft uses the Dinkumware one.
With your above example you're also missing several important pieces of information - the types you describe (Server::StringArray and Server::KeyValueMap) are standard library containers. OK fine, but standard library containers of what? These containers are templates and unless you know the exact types these templates have been instantiated with, you're a little stuck.
Is this DLL intended to be called from C++ at all? The fact that it export a JNI interface suggests that it might not be in the first place. Does it export any other public symbols apart from those that are of the format _Java_...?
Of course if there is no other way in and you must use C++ instead of Java, you might want to look into embedding a JVM into your C++ app and use that to call through to the C++ dll. It's not what I'd call an elegant solution but it might well work.
I don't quite understand the use of C++ standard library data types here. How can Java code provide a std::map argument? Are the arguments you pass in always just "opaque" values you would get as output from a previous call to the library? That's the only way you're going to be able to make it work from code under a different runtime.
Anyway...
When you make a JNI module, you run javah.exe and it generates a header file with declarations like:
JNIEXPORT void JNICALL Java_Native_HelloWorld(JNIEnv *, jobject);
Do you have any such header file for the module?
These symbols are exported as extern "C" if I recall correctly, so if you can get the correct signatures, you should have no issues with name mangling or incompatible memory allocators, etc..
The "#20" at the end of the method signature means that the function is declared "stdcall" and that 20 bytes are put on the stack when the function is called. All these methods should start with a JNIEnv* and a jobject, these will total 8 bytes I believe, on a 32-bit environment, so that leaves 12 bytes of parameters you will need to know in order to generate a correct function prototype.
Once you figure out what the parameters are, you can generate something like this:
typedef void (__stdcall *X)(JNIEnv *, jobject, jint i, jboolean b);
Then, you can cast the result of GetProcAddress to an X and call it from your C++ code.
X x = (X)GetProcAddress(module, "name");
if (x) x(params...);
Unfortunately, what you have doesn't quite look like what I have seen in the past. I am used to having to deal with Java data types from C/C++ code, but it looks like this module is dealing with C++ data types in Java code, so I don't know how relevant any of my experience is. Hopefully this is some help, at least.
Related
Now, I implemented a factory class to dynamically create class with a idenification string, please see the following code:
void IOFactory::registerIO()
{
Register("NDAM9020", []() -> IOBase * {
return new NDAM9020();
});
Register("BK5120", []() -> IOBase * {
return new BK5120();
});
}
std::unique_ptr<IOBase> IOFactory::createIO(std::string ioDeviceName)
{
std::unique_ptr<IOBase> io = createObject(ioDeviceName);
return io;
}
So we can create the IO class with the registered name:
IOFactory ioFactory;
auto io = ioFactory.createIO("BK5120");
The problem with this method is if we add another IO component, we should add another register code in registerIO function and compile the whole project again. So I was wondering if I could dynamically register class from a configure file(see below) at runtime.
io_factory.conf
------------------
NDAM9020:NDAM9020
BK5120:BK5120
------------------
The first is identification name and the second is class name.
I have tried with Macros, but the parameter in Macros cann't be string. So I was wondering if there is some other ways. Thanks for advance.
Update:
I didn't expect so many comments and answers, Thank you all and sorry for replying late.
Our current OS is Ubuntu16.04 and we use the builtin compiler that is gcc/g++5.4.0, and we use CMake to manage the build.
And I should mention that it is not a must that I should register class at runtime period, it is also OK if there is a way to do this in compile period. What I want is just avoiding the recompiling when I want to register another class.
So I was wondering if I could dynamically register class from a configure file(see below) at runtime.
No. As of C++20, C++ has no reflection features allowing it. But you could do it at compile time by generating a simple C++ implementation file from your configuration file.
How to dynamically register class in a factory class at runtime period with c++
Read much more about C++, at least a good C++ programming book and see a good C++ reference website, and later n3337, the C++11 standard. Read also the documentation of your C++ compiler (perhaps GCC or Clang), and, if you have one, of your operating system. If plugins are possible in your OS, you can register a factory function at runtime (by referring to to that function after a plugin providing it has been loaded). For examples, the Mozilla firefox browser or recent GCC compilers (e.g. GCC 10 with plugins enabled), or the fish shell, are doing this.
So I was wondering if I could dynamically register class from a configure file(see below) at runtime.
Most C++ programs are running under an operating system, such as Linux. Some operating systems provide a plugin mechanism. For Linux, see dlopen(3), dlsym(3), dlclose(3), dladdr(3) and the C++ dlopen mini-howto. For Windows, dive into its documentation.
So, with a recent C++ implementation and some recent operating systems, y ou can register at runtime a factory class (using plugins), and you could find libraries (e.g. Qt or POCO) to help you.
However, in pure standard C++, the set of translation units is statically known and plugins do not exist. So the set of functions, lambda-expressions, or classes in a given program is finite and does not change with time.
In pure C++, the set of valid function pointers, or the set of valid possible values for a given std::function variable, is finite. Anything else is undefined behavior. In practice, many real-life C++ programs accept plugins thru their operating systems or JIT-compiling libraries.
You could of course consider using JIT-compiling libraries such as asmjit or libgccjit or LLVM. They are implementation specific, so your code won't be portable.
On Linux, a lot of Qt or GTKmm applications (e.g. KDE, and most web browsers, e.g. Konqueror, Chrome, or Firefox) are coded in C++ and do load plugins with factory functions. Check with strace(1) and ltrace(1).
The Trident web browser of MicroSoft is rumored to be coded in C++ and probably accepts plugins.
I have tried with Macros, but the parameter in Macros can't be string.
A macro parameter can be stringized. And you could play x-macros tricks.
What I want is just avoiding the recompiling when I want to register another class.
On Ubuntu, I recommend accepting plugins in your program or library
Use dlopen(3) with an absolute file path; the plugin would typically be passed as a program option (like RefPerSys does, or like GCC does) and dlopen-ed at program or library initialization time. Practically speaking, you can have lots of plugins (dozen of thousands, see manydl.c and check with pmap(1) or proc(5)). The dlsym(3)-ed C++ functions in your plugins should be declared extern "C" to disable name mangling.
A single C++ file plugin (in yourplugin.cc) can be compiled with g++ -Wall -O -g -fPIC -shared yourplugin.cc -o yourplugin.so and later you would dlopen "./yourplugin.so" or an absolute path (or configure suitably your $LD_LIBRARY_PATH -see ld.so(8)- and pass "yourplugin.so" to dlopen). Be also aware of Rpath.
Consider also (after upgrading your GCC to GCC 9 at least, perhaps by compiling it from its source code) using libgccjit (it is faster than generating temporary C++ code in some file and compiling that file into a temporary plugin).
For ease of debugging your loaded plugins, you might be interested by Ian Taylor's libbacktrace.
Notice that your program's global symbols (declared as extern "C") can be accessed by name by passing a nullptr file path to dlopen(3), then using dlsym(3) on the obtained handle. You want to pass -rdynamic -ldl when linking your program (or your shared library).
What I want is just avoiding the recompiling when I want to register another class.
You might registering classes in a different translation unit (a short one, presumably). You could take inspiration from RefPerSys multiple #include-s of its generated/rps-name.hh file. Then you would simply recompile a single *.cc file and relink your entire program or library. Notice that Qt plays similar tricks in its moc, and I recommend taking inspiration from it.
Read also J.Pitrat's book on Artificial Beings: the Conscience of a Conscious Machine ISBN which explains why a metaprogramming approach is useful. Study the source code of GCC (or of RefPerSys), use or take inspiration from SWIG, ANTLR, GNU bison (they all generate C++ code) when relevant
You seem to have asked for more dynamism than you actually need. You want to avoid the factory itself having to be aware of all of the classes registered in it.
Well, that's doable without going all the way runtime code generation!
There are several implementations of such a factory; but I am obviously biased in favor of my own: einpoklum's Factory class (gist.github.com)
simple example of use:
#include "Factory.h"
// we now have:
//
// template<typename Key, typename BaseClass, typename... ConstructionArgs>
// class Factory;
//
#include <string>
struct Foo { Foo(int x) { }; }
struct Bar : Foo { Bar(int x) : Foo(x) { }; }
int main()
{
util::Factory<std::string, Foo, int> factory;
factory.registerClass<Bar>("key_for_bar");
auto* my_bar_ptr factory.produce("key_for_bar");
}
Notes:
The std::string is used as a key; you could have a factory with numeric values as keys instead, if you like.
All registered classes must be subclasses of the BaseClass value chosen for the factory. I believe you can change the factory to avoid that, but then you'll always be getting void *s from it.
You can wrap this in a singleton template to get a single, global, static-initialization-safe factory you can use from anywhere.
Now, if you load some plugin dynamically (see #BasileStarynkevitch's answer), you just need that plugin to expose an initialization function which makes registerClass() class calls on the factory; and call this initialization function right after loading the plugin. Or if you have a static-initialization safe singleton factory, you can make the registration calls in a static-block in your plugin shared library - but be careful with that, I'm not an expert on shared library loading.
Definetly YES!
Theres an old antique post from 2006 that solved my life for many years. The implementation runs arround having a centralized registry with a decentralized registration method that is expanded using a REGISTER_X macro, check it out:
https://web.archive.org/web/20100618122920/http://meat.net/2006/03/cpp-runtime-class-registration/
Have to admit that #einpoklum factory looks awesome also. I created a headeronly sample gist containing the code and a sample:
https://gist.github.com/h3r/5aa48ba37c374f03af25b9e5e0346a86
Since I am fairly new to Objective-C programming language, I'm facing a huge problem: how can I call a method of my application (made in Objective-C) from my dynamically loaded library (made in C++), by passing it some parameters?
I've been digging the web for a couple of days with no luck. The only thing I found is about IMP, but I'm not sure if that's what I need.
You actually have a plethora of options here.
If the dylib is one of your own, you can compile it as Objective-C++ for Mac OS X and #ifdef the objective-C calls (or not if you are only targeting Mac OS)
Both Obj-C and C++ can make use of C interfaces, so you could write an adapter layer in C (remember Obj-c is a strict superset of C) and expose it for the dylib to call the C functions which then call the Obj-C code.
You can use Blocks, which work in C, C++, and of course Obj-C
you can include the objective-c runtime (see documentation) and muck with that (This is where you would use the *IMP thing you mentioned).
Yet another option might be to use Objective C++ from the Cocoa side to setup C++ objects and expose those to the dylib. How you would go about this really depends on what the library is and how it is used etc; we need to know more about the dylib and how it is being used to elaborate on this.
Since you specifically mention using an IMP lets talk a bit more in depth about that. The declaration is typedef void (*IMP)(id self, SEL _cmd, ...); which you can see takes a pointer to an Obj-C objects, and a SEL (selector), which is just a special C-String representation of the method. You can read more about both SEL and IMP in the documentation.
You can also make use of the runtime's C functions such as objc_msgSend to call a method by passing a pointer to the object and a SEL just like with IMP.
This should be enough information to get you started. Thanks for this question BTW, I never really sat down and thought about all the possible ways to combine C++ with Objective-C before. Odds are I even missed something ;)
You can use objective c runtime
include <objc/runtime.h>
objc_msgSend(id, SEL, arg0, ...)
where
id - is the object where you want to send message
SEL - is struct pointer, describing message you send.
arg0,... are the arguments that you pass to selector.
For more understanding of runtime, see the source code http://www.opensource.apple.com/source/objc4/
Also you can cast IMP address, and call the function.
int(* foo)(id, SEL, arg) = IMP;
foo(someObject, #selector(someMessage), arg);
I have a list of functions in a text file that I'd like to expose to LLVM for its execution engine at run time, I'm wondering if its possible to find pointers to the functions at runtime rather than hard code in all the GlobalMappings by hand as I'd probably like to add in more later. For example:
// File: InternalFunctions.txt
PushScreen
PopScreen
TopScreen
// File: ExposeEngine.cpp
// Somehow figure out the address of the function specified in a string
void* addy = magicAddress("PushScreen");
jit->addGlobalMapping(llvmfunction, addy);
If this is possible I love to know how to do it, as I am trying to write my game engine by jit-ing c++. I was able to create some results earlier, but I had to hard-code in the mappings. I noticed that Gtk uses something along the lines of what I'm asking. When you use glade and provide a signal handler, the program you build in c will automatically find the function in your executable referenced by the string provided in the glade file. If getting results requires me to look into this Gtk thing I'd be more than happy to, but I don't know what feature or part of the api deals with that - I've already tried to look it up. I'd love to hear suggestions or advice.
Yes, you can do this. Look at the man pages for dlopen() and dlsym(): these functions are standard on *nix systems and let you look up symbols (functions or variables) by name. There is one significant issue, which is that C++ function names are usually "mangled" to encode type information. A typical way around this is to define a set of wrapper functions in an extern "C" {} block: these will be non-member, C-style functions which can then call into your C++ code. Their names will not be mangled, making them easy to look up using dlsym().
This is a pretty standard way that some plugin architectures work. Or at least used to work, before everyone started using interpreted languages!
I would like to accomplish a simple task:
I want to produce a C++/CLI lib that calls some .NET routines, and exposes 1 static method that I can call from purely native C++ app. I want to statically link to that lib from my native app. The signature of the method that the native C++ app would call should be
void (unsigned char* data, int length, _TCHAR* args[])
I am pretty new to C++/CLI and C++ in general. So, can you please help me and answer these questions:
For my tiny C++/CLI project, I assume the type needs to be class library with lib as output, targeting vc9 runtime (so that I can be sure it is available on end users PCs). Am I correct in this assumption?
Can you provide an example of the outline of the method with that signature I will need to write in my C++/CLI project. In particular how should I do conversion to CLR types properly (i.e. byte[], int32, and string)? And how do I decorate that method with something like "extern "C" __declspec(dllexport)" to make this method callable from my native C++ app?
How would I separate the code between cpp and h files in C++/CLI properly?
Finally how would I actually call it from my native app once I add lib reference?
Thank you.
Class library: correct, but you'll also need to change your project's configuration type from Dynamic Library to Static Library. I'm not sure what you mean by 'targeting vc9 runtime' – you need to target the same runtime as the native code that will be using your static library.
Since this is a static library, no __declspec(dllexport) is needed. If you want to know how to do conversion to .NET types, you'll need to post what your code is actually doing. In general, you'll want Marshal::Copy to copy a C-array into a .NET array, and marshal_as<> to copy C-strings into .NET strings, but there's still the question as to whether that data is intended to be mutable and needs to be marshaled back to native types before returning...
The exact same as in C++ – declaration in a header, definition in a source file.
The exact same as any other function – #include the header containing the declaration and call the function.
I'm trying to explicitly link with a DLL. No other resources is available except the DLL file itself and some documentation about the classes and its member functions.
From the documentation, each class comes with its own
member typedef
example: typedef std::map<std::string,std::string> Server::KeyValueMap, typedef std::vector<std::string> Server::String Array
member enumeration
example: enum Server::Role {NONE,HIGH,LOW}
member function
example: void Server::connect(const StringArray,const KeyValueMap), void Server::disconnect()
Implementing the codes from google search, i manage to load the dll can call the disconnect function..
dir.h
LPCSTR disconnect = "_Java_mas_com_oa_rollings_as_apiJNI_Server_1disconnect#20";
LPCSTR connect =
"_Java_mas_com_oa_rollings_as_apiJNI_Server_1connect#20";
I got the function name above from depends.exe. Is this what is called decorated/mangled function names in C++?
main.cpp
#include <iostream>
#include <windows.h>
#include <tchar.h>
#include "dir.h"
typedef void (*pdisconnect)();
int main()
{
HMODULE DLL = LoadLibrary(_T("server.dll"));
pdisconnect _pdisconnect;`
if(DLL)
{
std::cout<< "DLL loaded!" << std::endl;
_disconnect = (pdisconnect)GetProcAddress(DLL,disconnect);
if(_disconnect)
{
std::cout << "Successful link to function in DLL!" << std::endl;
}
else
{
std::cout<< "Unable to link to function in DLL!" << std::endl;
}
}
else
{
std::cout<< "DLL failed to load!" << std::endl;
}
FreeLibrary (DLL);
return 0;}
How do i call (for example) the connect member function which has the parameter datatype declared in the dll itself?
Edit
more info:
The DLL comes with an example implementation using Java. The Java example contains a Java wrapper generated using SWIG and a source code.
The documentation lists all the class, their member functions and also their datatypes. According to the doc, the list was generated from the C++ source codes.(??)
No other info was given (no info on what compiler was used to generate the DLL)
My colleague is implementing the interface using Java based on the Java example given, while I was asked to implement using C++. The DLL is from a third party company.
I'll ask them about the compiler. Any other info that i should get from them?
I had a quick read through about JNI but i dont understand how it's implemented in this case.
Update
i'm a little confused... (ok, ok... very confused)
Do i call(GetProcAddress) each public member function separately only when i want to use them?
Do i create a dummy class that imitates the class in the dll. Then inside the class definition, i call the equivalent function from the DLL? (Am i making sense here?) fnieto, is this what you're showing me at the end of your post?
Is it possible to instantiate the whole class from the DLL?
I was trying to use the connect function described in my first post. From the Depends.exe DLL output,
std::map // KeyValueMap has the following member functions: del, empty, get, has_1key,set
std::vector // StringArray has the following member functions: add, capacity, clear, get, isEMPTY, reserve, set, size
which is different from the member functions of map and vector in my compiler (VS 2005)...
Any idea? or am i getting the wrong picture here...
Unless you use a disassembler and try to figure out the paramater types from assemly code, you can't. These kind of information is not stored in the DLL but in a header file coming with the DLL. If you don't have it, the DLL is propably not meant to be used by you.
I would be very careful if I were you: the STL library was not designed to be used across compilation boundaries like that.
Not that it cannot be done, but you need to know what you are getting into.
This means that using STL classes across DLL boundaries can safely work only if you compile your EXE with the same exact compiler and version, and the same settings (especially DEBUG vs. RELEASE) as the original DLL. And I do mean "exact" match.
The C++ standard STL library is a specification of behavior, not implementation. Different compilers and even different revisions of the same compiler can, and will, differ on the code and data implementations. When your library returns you an std::map, it's giving you back the bits that work with the DLL's version of the STL, not necessarily the STL code compiled in your EXE.
(and I'm not even touching on the fact that name mangling can also differ from compiler to compiler)
Without more details on your circumstances, I can't be sure; but this can be a can of worms.
In order to link with a DLL, you need:
an import library (.LIB file), this describes the relation between C/C++ names and DLL exports.
the C/C++ signatures of the exported items (usually functions), describing the calling convention, arguments and return value. This usually comes in a header file (.H).
From your question it looks like you can guess the signatures (#2), but you really need the LIB file (#1).
The linker can help you generate a LIB from a DLL using an intermediate DEF.
Refer to this question for more details: How to generate an import library from a DLL?
Then you need to pass the .lib as an "additional library" to the linker. The DLL must be available on the PATH or in the target folder.