I have some trouble figuring out why the following crashes (MSVC9):
//// the following compiles to A.dll with release runtime linked dynamically
//A.h
class A {
__declspec(dllexport) std::string getString();
};
//A.cpp
#include "A.h"
std::string A::getString() {
return "I am a string.";
}
//// the following compiles to main.exe with debug runtime linked dynamically
#include "A.h"
int main() {
A a;
std::string s = a.getString();
return 0;
} // crash on exit
Obviously (?) this is due to the different memory models for the executable and DLL. Could it be that the string A::getString() returns is being allocated in A.dll and freed in main.exe?
If so, why - and what would be a safe way to pass strings between DLLs (or executables, for that matter)? Without using wrappers like shared_ptr with a custom deleter.
This isn't actually being caused by differing heap implementations - the MSVC std::string implementation doesn't use dynamically allocated memory for strings that small (it uses the small string optimization). The CRTs do need to match, but that isn't what bit you this time.
What's happening is that you're invoking undefined behaviour by violating the One Definition Rule.
The release and debug builds will have different preprocessor flags set, and you'll find that std::string has a different definition in each case. Ask your compiler what sizeof(std::string) is - MSVC10 tells me that it's 32 in a debug build and 28 in a release build (this isn't padding - 28 and 32 are both 4 bytes` boundaries).
So what's happening? Variable s is initialized using the debug version of the copy constructor to copy a release version of std::string. The offsets of the member variables are different between the versions, so you copy garbage. The MSVC implementation effectively stores begin and end pointers - you've copied garbage into them; because they're no longer null, the destructor tries to free them and you get an access violation.
Even if the heap implementations were the same it would crash, as you're freeing garbage pointers to memory that was never allocated in the first place.
In summary: the CRT versions need to match but so do the definitions - including the definitions in the standard library.
Could it be that the string
A::getString() returns is being
allocated in A.dll and freed in
main.exe?
Yes.
If so, why - and what would be a safe
way to pass strings between DLLs (or
executables, for that matter)? Without
using wrappers like shared_ptr with a
custom deleter.
Using a shared_ptr sounds like a sensible thing to me. Remember, as a rule of thumb, allocations and deallocations should be done by the same module to avoid glitches like these.
Exporting STL objects across dlls is at best a tricky pony. I suggest you check out this MSDN KB article first and this post.
You need to link to the same runtime lib (the DLL one), either debug or release, for every DLL in your app where memory is allocated in one and freed in another. (The reason for using the dynamically linked runtime lib is that then there will be one heap for your entire process, as opposed to one per dll/exe that links to the static one.)
This includes returning std::string and stl-containers by value, as that is what you do.
The reasons are twofold (updated section):
the classes have different layouts/sizes, so differently compiled code assumes data is in different places. Whoever created it first gets right, but the other one will cause a crash sooner or later.
the msvc heap-implementations are different in each runtime-lib which means that if you try to free a pointer in the heap that didn't allocate it, it will go bonkers. (This happens if the layouts are similar, i.e. where you outlive the first case.)
So, get your runtime libs straight, or stop freeing/allocating in different dlls (i.e. stop passing stuff by value).
In addition to what has been said above, make sure that the Platform Toolset (under Properties->General) is identical in both projects. Otherwise the string content at the arriving side might be bogus.
That happened to me when a console application project with v100 toolset version consumed a library that was set to v90.
This might be because the DLL and the EXE are compiled with different CRT settings. So when you pass a string, some resource conflict happens. Check your project settings for both the DLL and the executable.
Ensure both projects (App & Dll) are using one of the "Multi threaded DLL" runtime libraries & not the static version.
Properties --> C/C++ --> Code Generation--> (/MD or /MDd)
NOTE:
If you use any 3rd party libraries in your app then you may also need to recompile those, the linker will genrally notify you of this with mismatch/duplicate runtime errors.
Related
I created a .dll (with MFC static linking and Windows Runtime libraries statically linked as well) and I am linking to a library which uses boost for memory management (the library is PCL). Everything compiles ok with no errors, but I noticed that inside the library code, memory allocation is not working properly. For intance, the following line
indices_.reset (new std::vector<int>);
try {
indices_->resize (input_->points.size ());
}
allocates a new std::vector, does not throw any exception, but the vector is still empty after the resize function. Why should this be?
If i allocate the vector myself inside my own code for the DLL, the allocation works properly. But other errors arrise, such as strings that suddenly disappear (and the Visual Studio debugger shows "Error reading memory" when I hover on those strings).
I use static linking of the library to the DLL (and I am using static Runtime libraries, /MT).
What could be happening?
The static runtime libraries is the problem.
You can only interoperate memory with the same runtime library. If you're static linking then the DLL and EXE have their own distinct copies- which is very bad if you're trying to pass complex types between them.
You need to dynamically link the runtimes on both sides to get one copy of the CRT if you want to share complex objects like std::vector.
I believe that MFC has the same issue as the CRT- you need to share a single copy, not have a copy each.
Q. Why should this be?
The most obvious explanation would be that points.size () returns 0.
Otherwise, there would be UB involved (like when linking conflicting versions of libraries or ODR violations)
So, I have a C++ library with a statically linked copy of the MSVCRT. I want for anyone to be able to use my library with any version of the MSVC Runtime. What is the best way to accomplish this goal?
I'm already being quite careful with how things are done.
Memory never passes the DLL barrier to be freed
Runtime C++ objects aren't passed across barriers (ie, vectors, maps, etc.. unless they were created on that side of the barrier)
No file handles or resource handles are passed between barriers
Yet, I still have some simple code that causes heap corruption.
I have an object like so in my library:
class Foos
{
public: //There is an Add method, but it's not used, so not relevant here
DLL_API Foos();
DLL_API ~Foos();
private:
std::map<std::wstring, Foo*> map;
};
Foos::~Foos()
{
// start at the begining and go to the end deleting the data object
for(std::map<std::wstring, Foo*>::iterator it = map.begin(); it != map.end(); it++)
{
delete it->second;
}
map.clear();
}
And then I use it from my application like so:
void bar() {
Foos list;
}
After I call this function from anywhere, I get a debug warning about stack corruption. And If I actually let it run out, it actually does corrupt the stack and segfault.
My calling application is compiled with Visual Studio 2012 platform tools. The library is compiled using Visual Studio 2010 platform tools.
Is this just something I should absolutely not be doing, or am I actually violating the rules for using multiple runtimes?
Memory never passes the DLL barrier
But, it does. Many times in fact. Your application created the storage for the class object, in this case on the stack. And then passes a pointer to the methods in the library. Starting with the constructor call. That pointer is this inside the library code.
What goes wrong in a scenario like this one is that it didn't create the correct amount of storage. You got the VS2012 compiler to look at your class declaration. It uses the VS2012 implementation of std::map. Your library however was compiled with VS2010, it uses a completely different implementation of std::map. With an entirely different size. Huge changes thanks to C++11.
This is just complete memory corruption at work, code in your application that writes stack variables will corrupt the std::map. And the other way around.
Exposing C++ classes across module boundaries are filled with traps like that. Only ever consider it when you can guarantee that everything is compiled with the exact same compiler version and the exact same settings. No shortcuts on that, you can't mix Debug and Release build code either. Crafting the library so no implementation details are exposed is certainly possible, you have to abide by these rules:
Only expose pure interfaces with virtual methods, argument types must be simple types or interface pointers.
Use a class factory to create the interface instance
Use reference counting for memory management so it is always the library that releases.
Nail down core details like packing and calling convention with hard rules.
Never allow exceptions to cross the module boundary, only use error codes.
You'd be well on you way to write COM code by then, also the style you see used in for example DirectX.
map member variable is still created by application with some internal data allocated by application and not DLL (and they may use different implementations of map). As a rule of thumb don't use stack objects from DLLs, add something like Foos * CreateFoos() in your DLL.
Runtime C++ objects aren't passed across barriers (ie, vectors, maps,
etc.. unless they were created on that side of the barrier)
You are doing exactly that. Your Foos object is being created by the main program on the stack and then being used in the library. The object contains a map as a part of it...
When you compile the main program it looks at the header files etc to determine how much stack space to allocate for the Foos object. And the calls the constructor which is defined in the library... Which might have been expecting an entirely different layout/size of the object
It may not fit your needs, but don't forgot that implementing the whole thing in header files simplifies the problem (sort of) :-)
I have some code that I need to place in a common library dll. This code, a class CalibrationFileData, works perfectly fine when it's built as part of the current project. However, if CalibrationFileData is built in the common library, the program crashes, mentioning heap corruptions.
I have made sure that all allocations and deallocations occur within the class, with appropriate accessors, etc. Still, the problem won't go away. Just in case it makes any difference, I am sometimes passing vectors of pairs, definitely not plain old data, but the vector manipulation only occurs through the accessors, so there shouldn't be any allocation happening across modules.
Anything I'm missing?
Edit: The vectors are these:
std::vector<std::pair<CvPoint2D32f, CvPoint3D32f>>* extrinsicCorrespondences;
std::vector<int>* pointsPerImage;
I shouldn't need to worry about deep copies, since they're not heap allocated, right? Incidentally, I tried using pointers to vectors, as above, to sidestep the problem, but it didn't make a difference anyway.
Check the compile flags match between the library and the executable. For example, on Windows ensure you're using the same C Runtime Library (CRT) (/MD vs /MT). Check for warnings from the linker.
Are you certain that when you take ownership of the contents of the vector objects, within your methods, you are deep-copying them into your instance variables?
you should check deep copies inside vector object,it is pertain with deepcopy i think
Could it be different values defined for _SECURE_SCL in the two projects ?
It's likely you missed some piece of the client's code that tries to deallocate a memory that your DLL allocated or vice versa.
Perhaps the easiest thing to do would be to ensure that both client and the DLL use the same memory allocator. Not just the same kind, but the same actual "instance" of the allocator. On Visual C++, this is most easily achieved by both client and DLL using a "Multi-threaded Debug DLL (/MDd)" or "Multi-threaded DLL (/MD)" runtime library.
In the command line option, in visual studio remove this _SECURE_SCL. Mostly this crash is caused by _SECURE_SCL mismatch among the details. More details can be found here: http://kmdarshan.com/blog/?p=3830
You should distinguish between tightly coupled and loosely coupled DLLs.
Tightly coupled DLLs mean that
the DLL is built with the exact same compiler version, packing and
calling convention settings, library options as the application, and
both dynamically link to the runtime library (/MD compiler option).
This lets you pass objects back and forth including STL containers,
allocate DLL objects from inside the application, derive from base
classes in the other module, do just about everything you could
without using DLLs. The disadvantage is that you can no longer
deploy the DLL independently of the main application. Both must be
built together. The DLL is just to improve your process startup time
and working set, because the application can start running before
loading the DLL (using the /delayload linker option). Build times
are also faster than a single module, especially when whole program
optimization is used. But optimization doesn't take place across the
application-DLL boundary. And any non-trivial change will still
require rebuilding both.
In case of loosely coupled DLLs, then
When exporting DLL functions, it's best if they accept only integral
data types, i.e. int or pointers.
With reference, for example, to strings, then:
When you need to pass a string, pass it as a const char *. When you
need the DLL function to return a string, pass to the DLL a char *
pointer to a pre-allocated buffer, where the DLL would write the
string.
Finally, concerning memory allocation, then:
Never use memory allocated by the DLL outside of the DLL's own
functions, and never pass by value structures that have their own
constructor/destructor.
References
Heap corruption when returning from function inside a dll
Can I pass std::string to a DLL?
First_Layer
I have a win32 dll written in VC++6 service pack 6. Let's call this dll as FirstLayer. I do not have access to FirstLayer's source code but I need to call it from managed code. The problem is that FirstLayer makes heavy use of std::vector and std::string as function arguments and there is no way of marshaling these types into a C# application directly.
Second_Layer
The solution that I can think of is to first create another win32 dll written in VC++6 service pack 6. Let's call this dll as "SecondLayer". SecondLayer acts as a wrapper for FirstLayer. This layer contains wrapper classes for std::vector so std::vector is not exposed in all function parameters in this layer. Let's call this wrapper class for std::vector as StdVectorWrapper.
This layer does not make use of any new or delete operations to allocate or deallocate memory since this is handled by std::vector internally.
Third_Layer
I also created a VC++2005 class library as a wrapper for SecondLayer. This wrapper does all the dirty work of converting the unmanaged SecondLayer into managed code. Let's call this layer as "ThirdLayer".
Similar to SecondLayer, this layer does not make use of new and delete when dealing with StdVectorWrapper.
Fourth_Layer
To top it all, I created a C#2005 console application to call ThirdLayer. Let's call this C# console application as "FourthLayer".
Call Sequence Summary
FourthLayer(C#2005) -> ThirdLayer(VC++2005) -> SecondLayer(VC++6) -> FirstLayer(VC++6)
The Problem
I noticed that the "System.AccessViolationException: Attempted to read or write protected memory" exception is being thrown which I suspect to be due to SecondLayer's internal std::vector allocating memory which is illegal for ThirdLayer to access.
This is confirmed I think because when I recompile FirstLayer (simulated) and SecondLayer in VC++2005, the problem disappears completely. However, recompiling the production version of FirstLayer is not possible as I do not have the source code.
I have heard that in order to get rid of this problem, I need to write a shared memory allocator in C++ for SecondLayer's std::vector which is found in the StdVectorWrapper class. I do not fully understand why I need a shared memory allocator and how it works? Any idea?
Is there any readily available source code for this on the internet that I can just compile and use together with my code in SecondLayer?
Note that I am unable to use the boost library for this.
Each executable or dll links to a particular version of the c runtime library, which is what contains the implementation of new and delete. If two modules have different compilers (VC2005 vs VC6) or build settings (Debug vs Release) or other settings (Multithreaded runtime vs non-multithreaded runtime), then they will link to different c runtimes. That becomes a problem if memory allocated by one runtime is freed by a different runtime.
Now, if I'm not mistaken, templates (such as std::vector or std::string) can cause this problem to sneak in where it isn't immediately obvious. The issue comes from the fact that templates are compiled into each module separately.
Example: module 1 uses a vector (thus allocating memory), then passes it as a function parameter to module 2, and then module 2 manipulates the vector causing the memory to be deallocated. In this case, the memory was allocated using module 1's runtime and deallocated using module 2's runtime. If those runtimes are different, then you have a problem.
So given all that, you have two places for potential problems. One is between FirstLayer and SecondLayer if those two modules haven't been compiled with the exact same settings. The other is between SecondLayer and ThirdLayer if any memory is allocated in one and deallocated in the other.
You could write a couple more test programs to confirm which place(s) have problems.
To test FirstLayer-SecondLayer, copy the implementation of the SecondLayer functions into a VC6 program, write just enough code to call those functions in a typical manner, and link only against FirstLayer.
If that first test doesn't fail, or else once you've fixed it, then test SecondLayer-ThirdLayer: copy the ThirdLayer implementation into a VC2005 program, write the code to make typical calls, and link against SecondLayer.
I think you should look at a different solution architecture.
The binary code generated by VC6 stl vector and string is, I believe, different from the code generated by more recent version of VC because of the many stl upgrades. Because of this I don't think your architecture will work as the dlls will have two implementations of std::vector and std::string that are not binary compatible.
My suggested solution is to go back to VC6 and write a new wrapper dll for FirstLayer that exposes it via a pure C API -- this layer will need to be compiled using VC6sp6 to ensure it is binary compatible with FirstLayer. Then use PInvoke in Fourth_Layer to access the VC6 wrapper DLL.
I have found a solution for the problem. Basically, the StdVectorWrapper class which I wrote do not implement deep copy. So all I need to do is to add the following to the StdVectorWrapper class to implement deep copy.
Copy Constructor
Assignment Operator
Deconstructor
Edit: Alternative Solution
An even better solution would be to use clone_ptr for all the elements contained in std::vector as well as for std::vector itself. This eliminates the need for the copy constructor, assignment operator and deconstructor.
I have a Windows DLL that at one point, returns a pointer to a class that is new'ed in the DLL's code. The class itself is a very thin wrapper around another class private to the DLL.
The calling executable has no problem working with this class, everything works fine, with the exception that when the calling executable tries to delete this class, I get a RtlValidateHeap error.
The reasons behind the error makes sense, the exe is trying to free memory on the DLL's heap, which is normally a bad thing.
I've come up with a few possible solutions:
Override the class's new operator to
allocate its memory off of the
executable's heap (provided I can
even get at that heap space). The wrapper is very thin, so I would only be allocating a few bytes from the exe's heap.
Provide a special destruction
function for this class (yuck).
Tell the user not to destroy the
class and live with leaks (no way!)
Is there a 'normal' way of doing this?
Most libraries that use this sort of functionality, whether C or C++, actually go with your second option - specific functions to get and release objects. It allows distribution of binary DLLs that don't have to be compiled identically to the calling executable.
I've seen this behaviour is when one of the binaries was built with debug options (which uses a debug heap that, amongst other things, uses RtlValidateHeap) and the other binary was built as a release binary. It might be worth checking that you only use like with like...
By far the easiest solution is (for the Visual Studio users) to use /MD or /MDd everywhere. If the CRT itself is in a DLL, there's only one heap. Your app and your DLL will share it.
Are you loading the DLL dynamically at runtime (by calling LoadLibrary), or are you linking to using a .lib file during the link phase of your exe? If you're dynamically loading/unloading the DLL then when the delete happens is the DLL still loaded? This is a classic problem for plug-in type designs and the typical solution is something like your second proposal - specific functions to create and delete the objects in question.
Option 2 is pretty normal, eg, a Release() method that just does a "delete this".
Note that normally the DLL doesn't have it's own heap as such - most DLLs will share the process heap (see GetProcessHeap), but often the language runtime will decorate or add padding around allocated memory - so the pointer your dll ends up with is not the same pointer the heap will expect in a call to free.
The general rule is always release allocated memory with the matching free function, from within the same module.
There are some exceptions though - memory you know has been allocated directly with HeapAlloc from a known heap can be freed from another module. GlobalAlloc/GlobalFree works across modules. Strings allocated with SysAllocString can be freed from another module. Memory you know has been allocated by the same DLL version of the runtime can be freed from another module, but I'd stay away from this.
On the DLL side, I would use something along these lines:
dll.h: /* all exported, included by dll user */
class dll_class {
public:
virtual void foo() = 0;
}
dll_class* create_class();
void delete_class(dll_class*);
dll.cpp: /* implement the creator and destroyer */
dll_class* create_class() { return new dll_class_imp(); }
void delete_class(dll_class* pClass) { delete pClass; }
dll_imp.h: /* implement the actual dll_class logic */
class dll_class_imp : public dll_class {
public:
virtual void foo() {...}
}
Now, on the DLL user's side, dll.h will be included, and used like so:
dll_class* pClass = create_class();
pClass->foo();
delete_class(pClass);
this way, there's a clear distinction between the DLL's part and the user's part. Being an abstract class, the user cannot create a dll_class using new; and it only makes sense you delete object similarly to the way you've created them, i.e. with the exported functions.
What looks like will work for us was to override the new and delete operators for the problem class. It turned out that there were 3 situations that could happen:
EXE new's object, EXE delete's object
DLL new's object, DLL delete's object
DLL new's object, EXE delete's object
The new and delete operators ensure that the DLL would be allocating and freeing all memory allocations. So far, so good - testing shows that the operators are always being used and the memory is coming from the DLL's heap.