While working on a C++/CLI project to wrap a native C++ DLL, I've come across a native function that takes in a std::string. Something like the following:
class NativeApi
{
public:
ErrorCode readFile(std::string filename = "path.csv");
};
Inside my managed wrapper implementation I allocate a new instance of the native class and call this function:
ref class ManagedApi
{
private:
NativeApi *api;
public:
ManagedApi(): api(new NativeApi()) { }
void Read()
{
api->readFile("apath.csv") // or with nothing to use default value
}
}
When I run this, I get the MDA PinvokeStackImbalance complaining that this call has unbalanced the stack. I was surprised, since the only other time I ever got this MDA was from C# when calling conventions didn't match. I never saw this happen with C++/CLI, where presumably all the matching is done automatically by the compiler.
Has anyone ever saw this before? Googling came up empty. I've looked at the DLL signature and it looks something like:
?readFile#NativeApi##QAE?AW4ErrorCode##V?$basic_string#DU?$char_traits#D#std##V?$allocator#D#2##std###Z
This tells me that the function is there, and takes as a sole argument a basic_string, which should match the standard std::string typedef.
No idea what could possibly have gone wrong. I can make other calls to the native API that do not involve strings perfectly fine.
It is likely that there's a difference between the definition of std::string that you're using vs. what was used to compile the native C++ DLL. Even if the definitions are the same, the native DLL probably isn't using the same version of the C runtime as you are, so when your DLL allocates memory for the std::string, the Native DLL will try to call delete on it (when the string is destroyed at the end of the readFile method), and that call to delete will go to a different heap than was used to allocate the object!
If you want to make this work, you'll have to use the exact same version of the compiler as was used on the native DLL. Note that you'll be limited to the Release build of your project, as you don't have a native DLL that was compiled with the debug runtime.
The proper fix to this problem is to use raw types when calling across DLL boundaries (in this case, wchar_t*). If you can request a change to the native DLL, I would do that. If only raw types are used, then there's no issue with using different runtimes, and everything works the way it should.
Related
C++ class constructor can be inlined or not be inlined. However, I found a strange situation where only inline class constructor can avoid Visual Studio memory crash. The example is as follows:
dll.h
class _declspec(dllexport) Image
{
public:
Image();
virtual ~Image();
};
class _declspec(dllexport) Testimage:public Image
{
public:
Testimage();
virtual ~Testimage();
};
typedef std::auto_ptr<Testimage> TestimagePtr;
dll.cpp
#include "dll.h"
#include <assert.h>
Image::~Image()
{
std::cout<<"Image is being deleted."<<std::endl;
}
Image::Image()
{
}
Testimage::Testimage()
{
}
Testimage::~Testimage()
{
std::cout<<"Geoimage is being deleted."<<std::endl;
}
The dll library is compiled as a dynamic library, and it is statically linked to the C++ runtime library (Multi-threaded Debug (/MTd)). The executable program that runs the library is as follows:
int main()
{
TestimagePtr my_img(new Testimage());
return 0;
}
The executable program will invoke the dll library and it also statically links the runtime library. The problem I have is that when running the executable program the following error message appears:
However, when the class constructor in dll is inlined as the following codes show:
class _declspec(dllexport) Image
{
public:
Image();
virtual ~Image();
};
class _declspec(dllexport) Testimage:public Image
{
public:
Testimage()
{
}
virtual ~Testimage();
};
The crash will disappear. Could someone explain the reason behind? Thanks! By the way, I am using VC2010.
EDIT: The following situation also trigger the same crash
.
Situation 1
int main()
{
//TestimagePtr my_img(new Testimage());
Testimage *p_img;
p_img = new Testimage();
delete p_img;
return 0;
}
it is statically linked to the C++ runtime library (Multi-threaded Debug (/MTd)
This is a very problematic scenario in versions of Visual Studio prior to VS2012. The issue is that you have more than one version of the CRT loaded in your process. One used by your EXE, another used by the DLL. This can cause many subtle problems, and not so subtle problems like this crash.
The CRT has global state, stuff like errno and strtok() cannot work properly when that global state is updated by one copy of the CRT and read back by another copy. Relevant to your crash, a hidden global state variable is the heap that the CRT uses to allocate memory from. Functions like malloc() and ::operator new use that heap.
This goes wrong when objects are allocated by one copy of the CRT and released by another. The pointer that's passed to free() or ::operator delete belongs to the wrong heap. What happens next depends on your operating system. A silent memory leak in XP. In Vista and up, you program runs with the debug version of the memory manager enabled. Which triggers a breakpoint when you have a debugger attached to your process to tell you that there's a problem with the pointer. The dialog in your screenshot is the result. It isn't otherwise very clear to me how inlining the constructor could make a difference, the fundamental issue however is that your code invokes undefined behavior. Which has a knack for producing random outcomes.
There are two approaches available to solve this problem. The first one is the simple one, just build both your EXE and your DLL project with the /MD compile option instead. This selects the DLL version of the CRT. It is now shared by both modules and you'll only have a single copy of the CRT in your process. So there is no longer a problem with having one module allocating and another module releasing memory, the same heap is used.
This will work fine to solve your problem but can still become an issue later. A DLL tends to live a life of its own and may some day be used by another EXE that was built with a different version of the CRT. The CRT will now again not be shared since they'll use different versions of the DLL, invoking the exact same failure mode you are seeing today.
The only way to guarantee that this cannot happen is to design your DLL interface carefully. And ensure that there will never be a case where the DLL allocates memory that the client code needs to release. That requires giving up on a lot of C++ goodies. You for example can never write a function that returns a C++ object, like std::string. And you can never allow an exception to cross the module boundary. You are basically down to a C-style interface. Note how COM addresses this problem by using interface-based programming techniques and a class factory plus reference counting to solve the memory management problem.
VS2012 has a counter-measure against this problem, it has a CRT version that allocates from the default process heap. Which solves this particular problem, not otherwise a workaround for the global state issue for other runtime functions. And adds some new problems, a DLL compiled with /MT that gets unloaded that doesn't release all of its allocations now causes an unpluggable leak for example.
This is an ugly problem in C++, the language fundamentally misses an ABI specification that addresses problems like this. The notion of modules is entirely missing from the language specification. Being worked on today but not yet completed. Not simple to do, it is solved in other languages like Java and the .NET languages by specifying a virtual machine, providing a runtime environment where memory management is centralized. Not the kind of runtime environment that excites C++ programmers.
I tried to reproduce your problem in VC2010 and it doesn't crash. It works with a constructor inline or not. Your problem is probably not in what you write here.
Your project is too hard to open as it seams to have its file pathes set in absolute, probably because generated with CMake. (So the files are not found by the compiler).
The problem I see in your code is that you declare the exported classes with _declspec(dllexport) directly written.
You should have a #Define to do this, and the value should be _declspec(dllimport) when read from the exe compilation. Maybe the problem comes from that.
I am trying to migrate some static C++ libraries into DLLs with a C interface so I don't need to build a separate version of the library for every version of Visual Studio (i.e. CRT) we want to support. However, I do like the convenience of using STL objects for some of the function calls. I came up with something that seems to work, but was wondering if there may be some hidden things that I'm just not thinking of.
Here is what I came up with to get STL versions of functions while still maintaining Visual Studio independence.
Original library function:
//library.h
...
std::wstring GetSomeString();
...
StringGenerator* mStrGen; //assume forward declared for pimpl implementation
//library.cpp
std::wstring library::GetSomeString()
{
return mStrGen->GetString(); //returns a wstring;
}
First, I created a private function that would provide the C interface
//library.h
__declspec(dllexport) void GetSomeStringInternal(wchar_t* pSomeString);
//library.cpp
void library::GetSomeString(wchar_t*& pSomeString)
{
if(pSomeString!= nullptr) {
delete [] pSomeString; //assumes allocated by the DLL
}
std::wstring tmpString(mStrGen->GetString());
size_t stringLength(tmpString.size());
stringToReturn = new wchar_t[stringLength + 1];
wcscpy_s(pSomeString, stringLength + 1, tmpString.c_str());
}
Next, I added a private function that deallocates memory allocated by the DLL
//library.h
__declspec(dllexport) void FreeArray(void* arrayPtr);
//library.cpp
void library::FreeArray(void* arrayPtr)
{
if(arrayPtr) {
delete [] arrayPtr;
}
}
Finally, I converted the original C++ function returning a string into a function that calls the internal C interface function
//library.h
std::wstring GetSomeString()
{
std::wstring someString(L"");
wchar_t* pSomeString= NULL;
GetSomeStringInternal(pSomeString);
someString = pSomeString;
FreeArray(pSomeString);
return someString;
}
//library.cpp
//removed GetSomeString from cpp since it is defined in header
My thinking is that since the header will be compiled every time it is included, an application that uses a different version of the CRT will compile the function using its implementation of the CRT. All data passed into and out of the library uses a C interface to preserve compatibility and memory is allocated and freed by the library so you don't run into one version of the CRT trying to free memory from a different version.
It seems to perform as I intend:
The library can be used by programs compiled with multiple versions of Visual Studio.
There are no memory leaks or access violations
If I modify the code so that memory is allocated in the GetSomeString function in the header, I do get a memory access error when trying to free that memory.
The GetSomeString function is compiled by the library and included in the DLL, but it is never called since it is 1) not exported and 2) the compiling program will always choose its version since it is inlined.
Is there anything I am missing, or is this a valid way of going about providing a C++ interface to a library that is Visual Studio version independent?
Side note: I have run into some deletion issues if I have a program that uses a std::shared_ptr<library>, but haven't researched that issue enough and will probably have a follow up question on that problem.
One thing I could see becoming a problem is if you actually needed to pass large objects around by reference for performance reasons. You're dealing with the binary compatibility issues by copying all the data to and from a compatible format, which is fine until it becomes a performance issue.
I created c++ dll (using mingw) from code I wrote on linux (gcc), but somehow have difficulties using it in VC++. The dll basically exposes just one class, I created pure virtual interface for it and also factory function which creates the object (the only export) which looks like this:
extern "C" __declspec(dllexport) DeviceDriverApi* GetX5Driver();
I added extern "C" to prevent name mangling, dllexport is replaced by dllimport in actual code where I want to use the dll, DeviceDriverApi is the pure virtual interface.
Now I wrote simple code in VC++ which just call the factory function and then just tries to delete the pointer. It compiles without any problems but when I try to run it I get access violation error. If I try to call any method of the object I get access violation again.
When I compile the same code in MinGW (gcc) and use the same library, it runs without any problems. So there must be something (hehe, I guess many differences actually :)) between how VC++ code uses the library and gcc code.
Any ideas what?
Cheers,
Tom
Edit:
The code is:
DeviceDriverApi* x5Driver = GetX5Driver();
if (x5Driver->isConnected())
Console::WriteLine(L"Hello World");
delete x5Driver;
It's crashing when I try to call the method and when I try to delete the pointer as well. The object is created correctly though (the first line). There are some debug outputs when the object is created and I can see them before I get the access violation error.
You're using one compiler (mingw) for the DLL, and another (VC++) for the calling code.
You're calling a 'C' function, but returning a pointer to a C++ Object.
That will never work, because VTable layouts are almost guranteed to be incompatible. And, the DLL and app are probably using different memory managers, so you're doing new() with one and delete() with the other. Again, it just won't work.
For this to work the two compilers need to both support a standard ABI (Application Binary Interface). I don't think such a thing exists for Windows.
The best option is to expose all you DLL object methods and properties via C functions (including one to delete the object). You can the re-wrap into a C++ object on the calling end.
The two different compilers may be using different calling conventions. Try putting _cdecl before the function name in both the client code and the DLL code and recompiling both.
More info on calling conventions here: http://en.wikipedia.org/wiki/X86_calling_conventions
EDIT: The question was updated with more detail and it looks likely the problem is what Adrien Plisson describes at the end of his answer. You're creating an object in one module and freeing it in another, which is wrong.
(1) I suspect a calling covnention problem as well, though the simple suggestion by Leo doesn't seem to have helped.
Is isConnected virtual? It is possible that MinGW and VC++ use different implementations for a VTable, in which case, well, tough luck.
Try to see how far you get with the debugger: does it crash at the call, or the return? Do you arrive at invalid code? (If you know to read assembly, that usually helps a lot with these problems.)
Alternatively, add trace statements to the various methods, to see how far you get.
(2) For a public DLL interface, never free memory in the caller that was allocated by a callee (or vice versa). The DLL likely runs with a completely different heap, so the pointer is not known.
If you want to rely on that behavior, you need to make sure:
Caller and Callee (i.e. DLL and main program, in your case) are compiled with the same version of the sam compiler
for all supported compilers, you have configured the compile options to ensure caller and callee use the same shared runtime library state.
So the best way is to change your API to:
extern "C" __declspec(dllexport) DeviceDriverApi* GetX5Driver();
extern "C" __declspec(dllexport) void FreeDeviceDriver(DeviceDriverApi* driver);
and, at caller site, wrap in some way (e.g. in a boost::intrusive_ptr).
try looking at the imported libraries from both your DLL and your client executable. (you can use the Dependency Viewer or dumpbin or any other tool you like). verify that both the DLL and the client code are using the same C++ runtime.
if it is not the case, you can indeed run into some issues since the way the memory is managed may be different between the 2, leading to a crash when freeing from one runtime a pointer allocated from another runtime.
if this is really your problem, try not destroying the pointer in your client executable, but rather declare and export a function in your DLL which will take care of destroying the pointer.
I have a task to interface with a dll from a third party company using C++.
The dll package comes with:
the dll itself
a sample Java implementation - consists of a java wrapper(library) generated using the SWIG tool and the the java source file
documentation that states all the public datatypes, enumeration and member functions.
My other colleague is using Java(based on the example in package) to interface with the dll while I'm asked to use C++. The Java example looks straight forward... just import the wrapper and instantiate any class described in the docs..
More info on the dll:
From the docs, it says the dll was programmed using C++
From a hexdump, it shows that it was compiled using VC90 (VS C++ 2008 right?) and something from Dinkumware.
From a depends.exe output, the functions seems to be wrapped under JNI. For example: _Java_mas_com_oa_rollings_as_apiJNI_Server_1disconnect#20
My dilemma:
The dll company is not changing anything in the dll and not providing any other info.
How do i use the member functions in the class from the dll?
I did some simple LoadLibrary() and GetProcAddress and manage to get the address of the public member functions.
But i dunno how to use the functions that has the datatype parameters defined in the dll. For example:
From the docs, the member function is defined as:
void Server::connect(const StringArray, const KeyValueMap) throw(std::invalid_argument,std::out_of_range)
typedef std::map Server::KeyValueMap
typedef std::vector Server::StringArray
how do i call that function in C++. The std::map and std::vector in my compiler (VS 2005) has different functions listing that the one in the dll. For example, from the depends.exe output:
std::map // KeyValueMap - del, empty, get, has_1key,set
std::vector // StringArray - add, capacity, clear, get, isEMPTY, reserve, set, size
Any advice/strategy on how i should solve this? Is it possible to simply instantiate the class like the Java example?
If you are trying to use VS 2005 to try and interface with a DLL that is built using VS2008, your attempts will be mostly doomed unless you can use a plain C interface. Given your description, this is not the case; The runtime libraries differ between VS2005 and VS2008 so there is little chance that the object layout has stayed the same between compilers. The 'something from Dinkumware' that you're referring to is most likely the C++ standard library as ISTR that Microsoft uses the Dinkumware one.
With your above example you're also missing several important pieces of information - the types you describe (Server::StringArray and Server::KeyValueMap) are standard library containers. OK fine, but standard library containers of what? These containers are templates and unless you know the exact types these templates have been instantiated with, you're a little stuck.
Is this DLL intended to be called from C++ at all? The fact that it export a JNI interface suggests that it might not be in the first place. Does it export any other public symbols apart from those that are of the format _Java_...?
Of course if there is no other way in and you must use C++ instead of Java, you might want to look into embedding a JVM into your C++ app and use that to call through to the C++ dll. It's not what I'd call an elegant solution but it might well work.
I don't quite understand the use of C++ standard library data types here. How can Java code provide a std::map argument? Are the arguments you pass in always just "opaque" values you would get as output from a previous call to the library? That's the only way you're going to be able to make it work from code under a different runtime.
Anyway...
When you make a JNI module, you run javah.exe and it generates a header file with declarations like:
JNIEXPORT void JNICALL Java_Native_HelloWorld(JNIEnv *, jobject);
Do you have any such header file for the module?
These symbols are exported as extern "C" if I recall correctly, so if you can get the correct signatures, you should have no issues with name mangling or incompatible memory allocators, etc..
The "#20" at the end of the method signature means that the function is declared "stdcall" and that 20 bytes are put on the stack when the function is called. All these methods should start with a JNIEnv* and a jobject, these will total 8 bytes I believe, on a 32-bit environment, so that leaves 12 bytes of parameters you will need to know in order to generate a correct function prototype.
Once you figure out what the parameters are, you can generate something like this:
typedef void (__stdcall *X)(JNIEnv *, jobject, jint i, jboolean b);
Then, you can cast the result of GetProcAddress to an X and call it from your C++ code.
X x = (X)GetProcAddress(module, "name");
if (x) x(params...);
Unfortunately, what you have doesn't quite look like what I have seen in the past. I am used to having to deal with Java data types from C/C++ code, but it looks like this module is dealing with C++ data types in Java code, so I don't know how relevant any of my experience is. Hopefully this is some help, at least.
Sooooo I'm writing a script interpreter. And basically, I want some classes and functions stored in a DLL, but I want the DLL to look for functions within the programs that are linking to it, like,
program dll
----------------------------------------------------
send code to dll-----> parse code
|
v
code contains a function,
that isn't contained in the DLL
|
list of functions in <------/
program
|
v
corresponding function,
user-defined in the
program--process the
passed argument here
|
\--------------> return value sent back
to the parsing function
I was wondering basically, how do I compile a DLL with gcc? Well, I'm using a windows port of gcc. Once I compile a .dll containing my classes and functions, how do I link to it with my program? How do I use the classes and functions in the DLL? Can the DLL call functions from the program linking to it? If I make a class { ... } object; in the DLL, then when the DLL is loaded by the program, will object be available to the program? Thanks in advance, I really need to know how to work with DLLs in C++ before I can continue with this project.
"Can you add more detail as to why you want the DLL to call functions in the main program?"
I thought the diagram sort of explained it... the program using the DLL passes a piece of code to the DLL, which parses the code, and if function calls are found in said code then corresponding functions within the DLL are called... for example, if I passed "a = sqrt(100)" then the DLL parser function would find the function call to sqrt(), and within the DLL would be a corresponding sqrt() function which would calculate the square root of the argument passed to it, and then it would take the return value from that function and put it into variable a... just like any other program, but if a corresponding handler for the sqrt() function isn't found within the DLL (there would be a list of natively supported functions) then it would call a similar function which would reside within the program using the DLL to see if there are any user-defined functions by that name.
So, say you loaded the DLL into the program giving your program the ability to interpret scripts of this particular language, the program could call the DLLs to process single lines of code or hand it filenames of scripts to process... but if you want to add a command into the script which suits the purpose of your program, you could say set a boolean value in the DLL telling it that you are adding functions to its language and then create a function in your code which would list the functions you are adding (the DLL would call it with the name of the function it wants, if that function is a user-defined one contained within your code, the function would call the corresponding function with the argument passed to it by the DLL, the return the return value of the user-defined function back to the DLL, and if it didn't exist, it would return an error code or NULL or something). I'm starting to see that I'll have to find another way around this to make the function calls go one way only
This link explains how to do it in a basic way.
In a big picture view, when you make a dll, you are making a library which is loaded at runtime. It contains a number of symbols which are exported. These symbols are typically references to methods or functions, plus compiler/linker goo.
When you normally build a static library, there is a minimum of goo and the linker pulls in the code it needs and repackages it for you in your executable.
In a dll, you actually get two end products (three really- just wait): a dll and a stub library. The stub is a static library that looks exactly like your regular static library, except that instead of executing your code, each stub is typically a jump instruction to a common routine. The common routine loads your dll, gets the address of the routine that you want to call, then patches up the original jump instruction to go there so when you call it again, you end up in your dll.
The third end product is usually a header file that tells you all about the data types in your library.
So your steps are: create your headers and code, build a dll, build a stub library from the headers/code/some list of exported functions. End code will link to the stub library which will load up the dll and fix up the jump table.
Compiler/linker goo includes things like making sure the runtime libraries are where they're needed, making sure that static constructors are executed, making sure that static destructors are registered for later execution, etc, etc, etc.
Now as to your main problem: how do I write extensible code in a dll? There are a number of possible ways - a typical way is to define a pure abstract class (aka interface) that defines a behavior and either pass that in to a processing routine or to create a routine for registering interfaces to do work, then the processing routine asks the registrar for an object to handle a piece of work for it.
On the detail of what you plan to solve, perhaps you should look at an extendible parser like lua instead of building your own.
To your more specific focus.
A DLL is (typically?) meant to be complete in and of itself, or explicitly know what other libraries to use to complete itself.
What I mean by that is, you cannot have a method implicitly provided by the calling application to complete the DLLs functionality.
You could however make part of your API the provision of methods from a calling app, thus making the DLL fully contained and the passing of knowledge explicit.
How do I use the classes and functions in the DLL?
Include the headers in your code, when the module (exe or another dll) is linked the dlls are checked for completness.
Can the DLL call functions from the program linking to it?
Yes, but it has to be told about them at run time.
If I make a class { ... } object; in the DLL, then when the DLL is loaded by the program, will object be available to the program?
Yes it will be available, however there are some restrictions you need to be aware about. Such as in the area of memory management it is important to either:
Link all modules sharing memory with the same memory management dll (typically c runtime)
Ensure that the memory is allocated and dealloccated only in the same module.
allocate on the stack
Examples!
Here is a basic idea of passing functions to the dll, however in your case may not be most helpfull as you need to know up front what other functions you want provided.
// parser.h
struct functions {
void *fred (int );
};
parse( string, functions );
// program.cpp
parse( "a = sqrt(); fred(a);", functions );
What you need is a way of registering functions(and their details with the dll.)
The bigger problem here is the details bit. But skipping over that you might do something like wxWidgets does with class registration. When method_fred is contructed by your app it will call the constructor and register with the dll through usage off methodInfo. Parser can lookup methodInfo for methods available.
// parser.h
class method_base { };
class methodInfo {
static void register(factory);
static map<string,factory> m_methods;
}
// program.cpp
class method_fred : public method_base {
static method* factory(string args);
static methodInfo _methoinfo;
}
methodInfo method_fred::_methoinfo("fred",method_fred::factory);
This sounds like a job for data structures.
Create a struct containing your keywords and the function associated with each one.
struct keyword {
const char *keyword;
int (*f)(int arg);
};
struct keyword keywords[max_keywords] = {
"db_connect", &db_connect,
}
Then write a function in your DLL that you pass the address of this array to:
plugin_register(keywords);
Then inside the DLL it can do:
keywords[0].f = &plugin_db_connect;
With this method, the code to handle script keywords remains in the main program while the DLL manipulates the data structures to get its own functions called.
Taking it to C++, make the struct a class instead that contains a std::vector or std::map or whatever of keywords and some functions to manipulate them.
Winrawr, before you go on, read this first:
Any improvements on the GCC/Windows DLLs/C++ STL front?
Basically, you may run into problems when passing STL strings around your DLLs, and you may also have trouble with exceptions flying across DLL boundaries, although it's not something I have experienced (yet).
You could always load the dll at runtime with load library