Cross Compiler Library Interaction (Dynamic Loading) (plugins) - c++

This is so much a problem as verification of my reasoning.
Anyway I'm working on a complex project that will support plugins (libraries loaded dynamically at run time), Now I would like to be able to have the main program compiled with compiler X and the plugin compiled with compiler Y, and the plugin still work.
However I will need to pass complex data between the plugin and the main programs.
So my understanding is that standard functions are fine doesn't matter the compiler as long as I known the address they will execute the same basic code.
However when it comes to structs, 2 identical structs may not be the same due to alignment/packing method differing between compilers, however I can override this using #parama pack(n) on most compilers, and as long as this is the same the memory structure of these structs will match so they can be passed between the plugin and the main program.
Now then I believe this will also hold true for basic classes as long as there are no virtual functions, and all member variables are public.
However I can't relie on the calling conventions of member functions, so I must either map them though a standard function that takes the object as a parameter, or reimplemented them in the plugins namespace, most likely through a library its linked against.
Further more I should be able to implement operators and member functions for the structs/classes as long as they are implemented as above.
Hence I can use this for passing complex data between plugin and program, even though they were compiled with different compilers.
I'm I correct in my understanding here?

Yes, you seem to have most of the important points there. Essentially the DLLs must only expose a C-level interface, rather than many of the nifty features that C++ provides. (Although it's still possible to use C++ internally of course.)
One important point you didn't mention is that any memory allocated by a plugin must be deallocated by that same plugin. And similarly the main program must be the one to deallocated anything that it allocated. You can pass the memory addresses back and forth and use them fine, but deallocation is a special case because different compilers may use differing implementations of their heaps. Thus bad things could happen if one tries to clean up memory that it wasn't responsible for in the first place.

Related

Mixing versions of the MSVCRT

So, I have a C++ library with a statically linked copy of the MSVCRT. I want for anyone to be able to use my library with any version of the MSVC Runtime. What is the best way to accomplish this goal?
I'm already being quite careful with how things are done.
Memory never passes the DLL barrier to be freed
Runtime C++ objects aren't passed across barriers (ie, vectors, maps, etc.. unless they were created on that side of the barrier)
No file handles or resource handles are passed between barriers
Yet, I still have some simple code that causes heap corruption.
I have an object like so in my library:
class Foos
{
public: //There is an Add method, but it's not used, so not relevant here
DLL_API Foos();
DLL_API ~Foos();
private:
std::map<std::wstring, Foo*> map;
};
Foos::~Foos()
{
// start at the begining and go to the end deleting the data object
for(std::map<std::wstring, Foo*>::iterator it = map.begin(); it != map.end(); it++)
{
delete it->second;
}
map.clear();
}
And then I use it from my application like so:
void bar() {
Foos list;
}
After I call this function from anywhere, I get a debug warning about stack corruption. And If I actually let it run out, it actually does corrupt the stack and segfault.
My calling application is compiled with Visual Studio 2012 platform tools. The library is compiled using Visual Studio 2010 platform tools.
Is this just something I should absolutely not be doing, or am I actually violating the rules for using multiple runtimes?
Memory never passes the DLL barrier
But, it does. Many times in fact. Your application created the storage for the class object, in this case on the stack. And then passes a pointer to the methods in the library. Starting with the constructor call. That pointer is this inside the library code.
What goes wrong in a scenario like this one is that it didn't create the correct amount of storage. You got the VS2012 compiler to look at your class declaration. It uses the VS2012 implementation of std::map. Your library however was compiled with VS2010, it uses a completely different implementation of std::map. With an entirely different size. Huge changes thanks to C++11.
This is just complete memory corruption at work, code in your application that writes stack variables will corrupt the std::map. And the other way around.
Exposing C++ classes across module boundaries are filled with traps like that. Only ever consider it when you can guarantee that everything is compiled with the exact same compiler version and the exact same settings. No shortcuts on that, you can't mix Debug and Release build code either. Crafting the library so no implementation details are exposed is certainly possible, you have to abide by these rules:
Only expose pure interfaces with virtual methods, argument types must be simple types or interface pointers.
Use a class factory to create the interface instance
Use reference counting for memory management so it is always the library that releases.
Nail down core details like packing and calling convention with hard rules.
Never allow exceptions to cross the module boundary, only use error codes.
You'd be well on you way to write COM code by then, also the style you see used in for example DirectX.
map member variable is still created by application with some internal data allocated by application and not DLL (and they may use different implementations of map). As a rule of thumb don't use stack objects from DLLs, add something like Foos * CreateFoos() in your DLL.
Runtime C++ objects aren't passed across barriers (ie, vectors, maps,
etc.. unless they were created on that side of the barrier)
You are doing exactly that. Your Foos object is being created by the main program on the stack and then being used in the library. The object contains a map as a part of it...
When you compile the main program it looks at the header files etc to determine how much stack space to allocate for the Foos object. And the calls the constructor which is defined in the library... Which might have been expecting an entirely different layout/size of the object
It may not fit your needs, but don't forgot that implementing the whole thing in header files simplifies the problem (sort of) :-)

Are there any implementations of C++ that don't use vtables and vptrs? [duplicate]

C++ supports dynamic binding through virtual mechanism. But as I understand the virtual mechanism is an implementation detail of the compiler and the standard just specifies the behaviors of what should happen under specific scenarios. Most compilers implement the virtual mechanism through the virtual table and virtual pointer. This is not about implementation detail of virtual pointers and table. My questions are:
Are there any compilers which implement dynamic dispatch of virtual functions in any other way other than the virtual pointer and virtual table mechanism? As far as I have seen most (read G++, Microsoft Visual Studio) implement it through virtual table, pointer mechanism. So practically are there any other compiler implementations at all?
The sizeof of any class with just a virtual function will be size of an pointer (vptr inside this) on that compiler. So given that virtual pointer and TBL mechanism itself is compiler implementation, will this statement I made above be always true?
It is not true that vtable pointers in objects are always the most efficient. My compiler for another language used to use in-object pointers for similar reasons but no longer does: instead it uses a separate data structure which maps the object address to the required meta-data: in my system this happens to be shape information for use by the garbage collector.
This implementation costs a bit more storage for a single simple object, is more efficient for complex objects with many bases, and it is vastly more efficient for arrays, since only a single entry is required in the mapping table for all objects in the array. My particular implementation can also find the meta-data given a pointer to any point interior to the object.
The actual lookup is extremely fast, and the storage requirements very modest, because I am using the best data structure on the planet: Judy arrays.
I also know of no C++ compiler using anything other than vtable pointers, but it is not the only way. In fact, the initialisation semantics for classes with bases make any implementation messy. This is because the complete type has to see-saw around as the object is constructed. As a consequence of these semantics, complex mixin objects lead to massive sets of vtables being generated, large objects, and slow object initialisation. This probably isn't a consequence of the vtable technique as much as needing to slavishly follow the requirement that the run-time type of a subobject be correct at all times. Actually there's no good reason for this during construction, since constructors are not methods and can't sensibly use virtual dispatch: this isn't so clear to me for destruction since destructors are real methods.
To my knowledge, all C++ implementations use a vtable pointer, although it would be quite easy (and perhaps not so bad perf wise as you might think given caches) to keep a small type-index in the object (1-2 B) and subsequently obtain the vtable and type information with a small table lookup.
Another interesting approach might be BIBOP (http://foldoc.org/BIBOP) -- big bag of pages -- although it would have issues for C++. Idea: put objects of the same type on a page. Get a pointer to the type descriptor / vtable at the top of the page by simply and'ing off the less signficant bits of the object pointer. (Doesn't work well for objects on the stack, of course!)
Another other approach is to encode certain type tags/indices in the object pointers themselves. For example, if by construction all objects are 16-byte aligned, you can use the 4 LSBs to put a 4-bit type tag in there. (Not really enough.) Or (particularly for embedded systems) if you have guaranteed unused more-significant-bits in addresses, you can put more tag bits up there, and recover them with a shift and mask.
While both these schemes are interesting (and sometimes used) for other language implementations, they are problematic for C++. Certain C++ semantics, such as which base class virtual function overrides are called during (base class) object construction and destruction, drive you to a model where there is some state in the object that you modify as you enter base class ctors/dtors.
You may find my old tutorial on the Microsoft C++ object model implementation interesting.
http://www.openrce.org/articles/files/jangrayhood.pdf
Happy hacking!
I don't think there are any modern compilers with an approach other than vptr/vtable. Indeed, it would be hard to figure out something else that is not just plain inefficient.
However, there is still a pretty large room for design tradeoffs within that approach. Maybe especially regarding how virtual inheritance is handled. So it makes sense to make this implementation-defined.
If you are interested in this kind of stuff, I strongly suggest reading Inside the C++ Object Model.
sizeof class depends on the compiler. If you want portable code, don't make any assumptions.
Are there any compilers which implement Virtual Mechanism in any other way other than the virtual pointer and virtual table mechanism? As far as i have seen most(read g++,Microsoft visual studio) implement it through virtual table, pointer mechanism. So practically are there any other compiler implementations at all?
All current compilers that I know of use the vtable mechanism.
This is an optimization that's possible because C++ is statically type checked.
In some more dynamic languages there is instead a dynamic search up the base class chain(s), searching for an implementation of a member function that's called virtually, starting in the most derived class of the object. For example, that's how it worked in original Smalltalk. And the C++ standard describes the effect of a virtual call as if such a search had been used.
In Borland/Turbo Pascal in the 1990's such dynamic search was employed for finding handlers of Windows API "window messages". And I think possibly the same in Borland C++. It was in addition to the normal vtable mechanism, used solely for message handlers.
If it was used in Borland/Turbo C++ – I can't remember – then it was in support of a language extensions that allowed you to associate message id's with message handler functions.
The sizeof of any class with just a virtual function will be size of an pointer(vptr inside the this) on that compiler, So given that virtual ptr and tbl mechanism itself is compiler implementation, will this statement I made above be always true?
Formally no (even with assumption of vtable mechanism), it depends on the compiler. Since the standard doesn't require the vtable mechanism it says nothing about placement of vtable pointer in each object. And other rules let the compiler freely add padding, unused bytes, at the end.
But in practice perhaps. ;-)
However it's not something that you should rely on, or that you need to rely on. But in the other direction you can require this, for example if you're defining an ABI. Then any compiler that doesn't, simply doesn't conform to your requirement.
Cheers & hth.,
Are there any compilers which implement Virtual Mechanism in any other way other than the virtual pointer and virtual table mechanism? As far as i have seen most(read g++,Microsoft visual studio) implement it through virtual table, pointer mechanism. So practically are there any other compiler implementations at all?
None that I'm aware of C++ compilers using, though you might find it interesting to read about Binary Tree Dispatch. If you're interested in exploiting the expectation of virtual dispatch tables in any way, you should be aware that compilers can - where the types are known at compile time - sometimes resolve virtual function calls at compile time, so may not consult the table.
The sizeof of any class with just a virtual function will be size of an pointer(vptr inside the this) on that compiler, So given that virtual ptr and tbl mechanism itself is compiler implementation, will this statement I made above be always true?
Assuming no base classes with their own virtual members, and no virtual base classes, it's overwhelmingly likely to be true. Alternatives can be envisaged - such as whole-program analysis revealing only one member in the class heirarchy, and a switch to compile-time dispatch. If run-time dispatch is required, it's hard to imagine why any compiler would introduce further indirection. Still, the Standard deliberately doesn't stipulate these things precisely so that implementations can vary, or be varied in future.
In trying to imagine an alternative scheme, I have come up with the following, along the lines of Yttril's answer. As far as I'm aware, no compiler uses it!
Given a sufficiently large virtual address space and flexible OS memory allocation routines, it would be possible for new to allocate objects of different types in fixed, non-overlapping address ranges. Then the type of an object could be inferred quickly from its address using a right-shift operation, and the result used to index a table of vtables, thus saving 1 vtable pointer per object.
At first glance this scheme might seem to run into problems with stack-allocated objects, but this can be handled cleanly:
For each stack-allocated object, the compiler adds code that adds a record to a global array of (address range, type) pairs when the object is created and removes the record when it is destroyed.
The address range comprising the stack would map to a single vtable containing a large number of thunks that read the this pointer, scan the array to find the corresponding type (vptr) for the object at that address, and call the corresponding method in the vtable pointed to. (I.e. the 42nd thunk will call the 42nd method in the vtable -- if the most virtual functions used in any class is n, then at least n thunks are required.)
This scheme obviously incurs non-trivial overhead (at least O(log n) for the lookup) for virtual method calls on stack-based objects. In the absence of arrays or composition (containment within another object) of stack-based objects, a simpler and faster approach can be used in which the vptr is placed on the stack immediately before the object (note that it is not considered part of the object and does not contribute to its size as measured by sizeof). In this case thunks simply subtract sizeof (vptr) from this to find the correct vptr to use, and forward as before.
IIRC Eiffel uses a different approach and all overrides of a method end up merged and compiled in the same address with a prologue where the object type is checked (so every object must have a type ID, but it's not a pointer to a VMT). This for C++ would require of course that the final function is created at link time.
I don't know any C++ compiler that uses this approach, however.
I've never heard of or seen any compiler that uses any alternative implementation. The reason that vtables are so popular is because that not only is it the most efficient implementation, but it's also the easiest design and most obvious implementation.
On pretty much any compiler you care to use, it's almost certainly true. However, it's not guaranteed and not always true- you can't depend on it, even though it's pretty much always the case. Your favourite compiler could also alter it's alignment, increasing it's size, for funsies, without telling you. From memory, it can also insert whatever debug information and whatever it likes.
C++/CLI deviates from both assumptions. If you define a ref class, it doesn't get compiled into machine code at all; instead, the compiler compiles it into .NET managed code. In the intermediate language, classes are a built-in feature, and the set of virtual methods is defined in the metadata, rather than a method table.
The specific strategy to implement object layout and dispatch depends on the VM. In Mono, an object containing just one virtual method has not the size of one pointer, but needs two pointers in the MonoObject struct; the second one for the synchronization of the object. As this is implementation-defined and also not really useful to know, sizeof is not supported for ref classes in C++/CLI.
First, there were mentioned Borland's proprietary extension to C++, Dynamic Dispatch Virtual Tables (DDVT), and you can read something about it in a file named DDISPATC.ZIP. Borland Pascal had both virtual and dynamic methods, and Delphi introduced yet another "message" syntax, similar to dynamic, but for messages. At this point I'm not sure if Borland C++ had the same features. There was no multiple inheritance in either Pascal or Delphi, so Borland C++ DDVT might be different from either Pascal or Delphi.
Second, in 1990s and a bit earlier there was experimenting with different object models, and Borland was not the most advanced one. I personally think that shutting down IBM SOMobjects did a damage to the world that we all still suffering from. Before shutting down SOM there were experiments with Direct-to-SOM C++ compilers. So instead of C++'s way of invoking methods SOM is used. It is in many ways similar to C++ vtable, with several exceptions. First, to prevent fragile base class problem, programs do not use offsets inside of vtable, because they don't know this offset. It can change if base class introduces new methods. Instead, callers invoke a thunk created in runtime that has this knowledge in its assembly code. And there is one more difference. In C++, when multiple inheritance is used, an object can contain several VMTs IIRC. In contrast to C++, each SOM object has just one VMT, so dispatch code should be different from "call dword ptr [VMT+offset]".
There is a document related to SOM, Release-to-Release Binary Compatibility in SOM. You can find comparison of SOM with another projects I know little of, like Delta/C++ and Sun OBI. They solve a subset of problems that SOM solves, and by doing so they are also having somewhat tweaked invokation code.
I have recently found Visual Age C++ v3.5 for Windows compiler fragment enough to get things running and actually touch it. Most of users are not likely to get OS/2 VM just to play with DTS C++, but having Windows compiler is completely another matter. VAC v3.5 is the first and the last version to support Direct-to-SOM C++ feature. VAC v3.6.5 and v4.0 are not appropriate.
Download VAC 3.5 fixpak 9 from IBM FTP. This fixpak contain many files, so you don't even need to full compiler (I have 3.5.7 distro, but fixpak 9 was big enough to do some tests).
Unpack to e. g. C:\home\OCTAGRAM\DTS
Start command line and run subsequent commands there
Run: set SOMBASE=C:\home\OCTAGRAM\DTS\ibmcppw
Run: C:\home\OCTAGRAM\DTS\ibmcppw\bin\SOMENV.BAT
Run: cd C:\home\OCTAGRAM\DTS\ibmcppw\samples\compiler\dts
Run: nmake clean
Run: nmake
hhmain.exe and its dll are in different directories, so we must make them find each other somehow; since I was doing several experiments, I executed "set PATH=%PATH%;C:\home\OCTAGRAM\DTS\ibmcppw\samples\compiler\dts\xhmain\dtsdll" once, but you can just copy dll near to hhmain.exe
Run: hhmain.exe
I've got an output this way:
Local anInfo->x = 5
Local anInfo->_get_x() = 5
Local anInfo->y = A
Local anInfo->_get_y() = B
{An instance of class info at address 0092E318
}
Tony D's answer correctly points out that compilers are allowed to use whole-program analysis to replace a virtual function call with a static call to the unique possible function implementation; or to compile obj->method() into the equivalent of
if (auto frobj = dynamic_cast<FrequentlyOccurringType>(obj)) {
frobj->FrequentlyOccurringType::method(); // static dispatch on hot path
} else {
obj->method(); // vtable dispatch on cold path
}
Karel Driesen and Urs Hölzle wrote a really fascinating paper way back in 1996 in which they simulated the effect of perfect whole-program optimization on typical C++ applications: "The Direct Cost of Virtual Function Calls in C++". (The PDF is available for free if you Google for it.) Unfortunately, they only benchmarked vtable dispatch versus perfect static dispatch; they didn't compare it to binary tree dispatch.
They did point out that there are actually two kinds of vtables, when you're talking about languages (like C++) that support multiple inheritance. With multiple inheritance, when you call a virtual method that is inherited from the second base class, you need to "fix up" the object pointer so it points to an instance of the second base class. This fixup offset can be stored as data in the vtable, or it can be stored as code in a "thunk". (See the paper for more details.)
I believe all decent compilers these days use thunks, but it did take 10 or 20 years for that market penetration to reach 100%.

Are classes guaranteed to have the same organization in memory between program runs?

I'm attempting to implement a Save/Load feature into my small game. To accomplish this I have a central class that stores all the important variables of the game such as position, etc. I then save this class as binary data to a file. Then simply load it back for the loading function. This seems to work MOST of the time, but if I change certain things then try to do a save/load the program will crash with memory access violations. So, are classes guaranteed to have the same structure in memory on every run of the program or can the data be arranged at random like a struct?
Response to Jesus - I mean the data inside the class, so that if I save the class to disk, when I load it back, will everything fit nicely back.
Save
fout.write((char*) &game,sizeof Game);
Load
fin.read((char*) &game, sizeof Game);
Your approach is extremely fragile. With many restrictions, it can work. These restrictions are not worth subjecting your users (or yourself!) to in typical cases.
Some Restrictions:
Never refer to external memory (e.g. a pointer or reference)
Forbid ABI changes/differences. Common case: memory layout and natural alignment on 32 vs 64 will vary. The user will need a new 'game' for each ABI.
Not endian compatible.
Altering your type's layouts will break your game. Changing your compiler options can do this.
You're basically limited to POD data.
Use offsets instead of pointers to refer to internal data (This reference would be in contiguous memory).
Therefore, you can safely use this approach in extremely limited situations -- that typically applies only to components of a system, rather than the entire state of the game.
Since this is tagged C++, "boost - Serialization" would be a good starting point. It's well tested and abstracts many of the complexities for you.
Even if this would work, just don't do it. Define a file format at the byte-level and write sensible 'convert to file format' and 'convert from file format' functions. You'll actually know the format of the file. You'll be able to extend it. Newer versions of the program will be able to read files from older versions. And you'll be able to update your platform, build tools, and classes without fear of causing your program to crash.
Yes, classes and structures will have the same layout in memory every time your program runs., although I can't say if the standard enforces this. The machine code generated by C++ compilers use "hard-coded" offsets to access type fields, so they are fixed. Realistically, the layout will only change if you modify the C++ class definition (field sizes, order, virtual methods, etc.), compile with a different compiler or change compiler options.
As long as the type is POD and without pointer fields, it should be safe to simply dump it to a file and read it back with the exact same program. However, because of the above-mentionned concerns, this approach is quite inflexible with regard to versionning and interoperability.
[edit]
To respond to your own edit, do not do this with your "Game" object! It certainly has pointers to other objects, and those objects will not exist anymore in memory or will be elsewhere when you'll reload your file.
You might want to take a look at this.
Classes are not guaranteed to have the same structure in memory as pointers can point to different locations in memory each time a class is created.
However, without posting code it is difficult to say with certainty where the problem is.

Heap corruption when using DLL code

I have some code that I need to place in a common library dll. This code, a class CalibrationFileData, works perfectly fine when it's built as part of the current project. However, if CalibrationFileData is built in the common library, the program crashes, mentioning heap corruptions.
I have made sure that all allocations and deallocations occur within the class, with appropriate accessors, etc. Still, the problem won't go away. Just in case it makes any difference, I am sometimes passing vectors of pairs, definitely not plain old data, but the vector manipulation only occurs through the accessors, so there shouldn't be any allocation happening across modules.
Anything I'm missing?
Edit: The vectors are these:
std::vector<std::pair<CvPoint2D32f, CvPoint3D32f>>* extrinsicCorrespondences;
std::vector<int>* pointsPerImage;
I shouldn't need to worry about deep copies, since they're not heap allocated, right? Incidentally, I tried using pointers to vectors, as above, to sidestep the problem, but it didn't make a difference anyway.
Check the compile flags match between the library and the executable. For example, on Windows ensure you're using the same C Runtime Library (CRT) (/MD vs /MT). Check for warnings from the linker.
Are you certain that when you take ownership of the contents of the vector objects, within your methods, you are deep-copying them into your instance variables?
you should check deep copies inside vector object,it is pertain with deepcopy i think
Could it be different values defined for _SECURE_SCL in the two projects ?
It's likely you missed some piece of the client's code that tries to deallocate a memory that your DLL allocated or vice versa.
Perhaps the easiest thing to do would be to ensure that both client and the DLL use the same memory allocator. Not just the same kind, but the same actual "instance" of the allocator. On Visual C++, this is most easily achieved by both client and DLL using a "Multi-threaded Debug DLL (/MDd)" or "Multi-threaded DLL (/MD)" runtime library.
In the command line option, in visual studio remove this _SECURE_SCL. Mostly this crash is caused by _SECURE_SCL mismatch among the details. More details can be found here: http://kmdarshan.com/blog/?p=3830
You should distinguish between tightly coupled and loosely coupled DLLs.
Tightly coupled DLLs mean that
the DLL is built with the exact same compiler version, packing and
calling convention settings, library options as the application, and
both dynamically link to the runtime library (/MD compiler option).
This lets you pass objects back and forth including STL containers,
allocate DLL objects from inside the application, derive from base
classes in the other module, do just about everything you could
without using DLLs. The disadvantage is that you can no longer
deploy the DLL independently of the main application. Both must be
built together. The DLL is just to improve your process startup time
and working set, because the application can start running before
loading the DLL (using the /delayload linker option). Build times
are also faster than a single module, especially when whole program
optimization is used. But optimization doesn't take place across the
application-DLL boundary. And any non-trivial change will still
require rebuilding both.
In case of loosely coupled DLLs, then
When exporting DLL functions, it's best if they accept only integral
data types, i.e. int or pointers.
With reference, for example, to strings, then:
When you need to pass a string, pass it as a const char *. When you
need the DLL function to return a string, pass to the DLL a char *
pointer to a pre-allocated buffer, where the DLL would write the
string.
Finally, concerning memory allocation, then:
Never use memory allocated by the DLL outside of the DLL's own
functions, and never pass by value structures that have their own
constructor/destructor.
References
Heap corruption when returning from function inside a dll
Can I pass std::string to a DLL?

C++ : When do I need a shared memory allocator for std::vector?

First_Layer
I have a win32 dll written in VC++6 service pack 6. Let's call this dll as FirstLayer. I do not have access to FirstLayer's source code but I need to call it from managed code. The problem is that FirstLayer makes heavy use of std::vector and std::string as function arguments and there is no way of marshaling these types into a C# application directly.
Second_Layer
The solution that I can think of is to first create another win32 dll written in VC++6 service pack 6. Let's call this dll as "SecondLayer". SecondLayer acts as a wrapper for FirstLayer. This layer contains wrapper classes for std::vector so std::vector is not exposed in all function parameters in this layer. Let's call this wrapper class for std::vector as StdVectorWrapper.
This layer does not make use of any new or delete operations to allocate or deallocate memory since this is handled by std::vector internally.
Third_Layer
I also created a VC++2005 class library as a wrapper for SecondLayer. This wrapper does all the dirty work of converting the unmanaged SecondLayer into managed code. Let's call this layer as "ThirdLayer".
Similar to SecondLayer, this layer does not make use of new and delete when dealing with StdVectorWrapper.
Fourth_Layer
To top it all, I created a C#2005 console application to call ThirdLayer. Let's call this C# console application as "FourthLayer".
Call Sequence Summary
FourthLayer(C#2005) -> ThirdLayer(VC++2005) -> SecondLayer(VC++6) -> FirstLayer(VC++6)
The Problem
I noticed that the "System.AccessViolationException: Attempted to read or write protected memory" exception is being thrown which I suspect to be due to SecondLayer's internal std::vector allocating memory which is illegal for ThirdLayer to access.
This is confirmed I think because when I recompile FirstLayer (simulated) and SecondLayer in VC++2005, the problem disappears completely. However, recompiling the production version of FirstLayer is not possible as I do not have the source code.
I have heard that in order to get rid of this problem, I need to write a shared memory allocator in C++ for SecondLayer's std::vector which is found in the StdVectorWrapper class. I do not fully understand why I need a shared memory allocator and how it works? Any idea?
Is there any readily available source code for this on the internet that I can just compile and use together with my code in SecondLayer?
Note that I am unable to use the boost library for this.
Each executable or dll links to a particular version of the c runtime library, which is what contains the implementation of new and delete. If two modules have different compilers (VC2005 vs VC6) or build settings (Debug vs Release) or other settings (Multithreaded runtime vs non-multithreaded runtime), then they will link to different c runtimes. That becomes a problem if memory allocated by one runtime is freed by a different runtime.
Now, if I'm not mistaken, templates (such as std::vector or std::string) can cause this problem to sneak in where it isn't immediately obvious. The issue comes from the fact that templates are compiled into each module separately.
Example: module 1 uses a vector (thus allocating memory), then passes it as a function parameter to module 2, and then module 2 manipulates the vector causing the memory to be deallocated. In this case, the memory was allocated using module 1's runtime and deallocated using module 2's runtime. If those runtimes are different, then you have a problem.
So given all that, you have two places for potential problems. One is between FirstLayer and SecondLayer if those two modules haven't been compiled with the exact same settings. The other is between SecondLayer and ThirdLayer if any memory is allocated in one and deallocated in the other.
You could write a couple more test programs to confirm which place(s) have problems.
To test FirstLayer-SecondLayer, copy the implementation of the SecondLayer functions into a VC6 program, write just enough code to call those functions in a typical manner, and link only against FirstLayer.
If that first test doesn't fail, or else once you've fixed it, then test SecondLayer-ThirdLayer: copy the ThirdLayer implementation into a VC2005 program, write the code to make typical calls, and link against SecondLayer.
I think you should look at a different solution architecture.
The binary code generated by VC6 stl vector and string is, I believe, different from the code generated by more recent version of VC because of the many stl upgrades. Because of this I don't think your architecture will work as the dlls will have two implementations of std::vector and std::string that are not binary compatible.
My suggested solution is to go back to VC6 and write a new wrapper dll for FirstLayer that exposes it via a pure C API -- this layer will need to be compiled using VC6sp6 to ensure it is binary compatible with FirstLayer. Then use PInvoke in Fourth_Layer to access the VC6 wrapper DLL.
I have found a solution for the problem. Basically, the StdVectorWrapper class which I wrote do not implement deep copy. So all I need to do is to add the following to the StdVectorWrapper class to implement deep copy.
Copy Constructor
Assignment Operator
Deconstructor
Edit: Alternative Solution
An even better solution would be to use clone_ptr for all the elements contained in std::vector as well as for std::vector itself. This eliminates the need for the copy constructor, assignment operator and deconstructor.