I have an EXE file using a DLL file which is using another DLL file. This situation has arisen:
In DLL file 1:
class abc
{
static bool FindSubFolders(const std::string & sFolderToCheck,
std::vector< std::string > & vecSubFoldersFound);
}
In DLL file 2:
void aFunction()
{
std::vector<std::string> folders;
std::string sLocation;
...
abc::FindSubFolders(sLocation, folders)
}
In release mode, everything works fine. But in debug mode, I come up with an assertion failure in the destructor of one of the std::strings in the folders vector (when folders goes out of scope at the end of aFunction):
dbgheap.c : line 1274
/*
* If this ASSERT fails, a bad pointer has been passed in. It may be
* totally bogus, or it may have been allocated from another heap.
* The pointer MUST come from the 'local' heap.
*/
_ASSERTE(_CrtIsValidHeapPointer(pUserData));
I assume this is because the memory has been allocated on DLL file 1's heap, but is being freed in DLL file 2.
The comment in dbgheap.c seems pretty insistent that this is a problem.
Why is this such a problem, when it seems to work fine if I just ignore it? Is there a non-assertion-failing way of doing this?
As sean has already said, the release build simply ignores that delete statement, so the best you can hope for is a memory leak.
If you have control over how both DLL files are compiled, make sure to use the Multi-threaded Debug DLL (/MDd) or Multi-threaded DLL (/MD) settings for the runtime library. That way, both DLL files will use the same runtime system and share the same heap.
The downside is that you need to install the runtime system together with your application (Microsoft offers an installer for that). It will work fine on your development machine since Visual Studio installs that runtime system too, but on a freshly installed machine it will report missing DLL files.
Most likely, the release build has the same problem, but release builds don't assert. They just ignore the problem. You might never see an issue. Or you might see data corruption. Or you might see a crash. Maybe only your users will experience bugs that you are simply not able to reproduce.
Don't ignore CRT assertions.
You should always use the appropriate deallocator (the one that matches the allocator used to begin with). If you are using static CRT libraries in your DLL files, the DLL files are using different heaps. You can not deallocate memory across heaps. Allocate and deallocate a block of memory using the same heap.
If you are using shared CRT libraries in your DLL files, then they should be using the same heap and you can allocate in one DLL file and deallocate in another.
As other says, the problem can be solved by making sure that the CRT is shared between the two modules. But there are common scenarios where this contract is hard to enforce.
The reason is that making sure to link against a shared CRT will not work if the EXE and DLL do not link against the same CRT version (as in 6.0, 7.0, 8.0). For example if you take a DLL that has been built in VC6.0 and try to use it with an EXE build in VS2010 you will get the same issue as before. The two versions of CRT will be loaded in your process and will each use their own heap for allocation, regardless if your EXE and DLL use 'shared' CRT, they will not be the same.
A better practice for an API would be to make sure that objects allocated in one side are also destroyed on the same side. I know this sounds ugly but it is the only way to ensure that a DLL remains binary compatible.
This is only a problem if the application or one (or more) of the DLL files is linked against the static version of the standard library. This was solved about a decade ago by MS releasing the shared library version of the standard library. This is because each version of the standard library will build it own internal heap and thus with multiple heaps you must release the memory to the correct heap. By using the shared version of the standard library they all use the same heap.
It is standard practice nowadays for the application and all DLL files should be built to use the dynamic version of the standard library.
The only caveat to the above is if you create your own heap and allocate memory from this heap. But this is a very specialized procedure only done in rare situations (and if you understand enough to use it then you will not be in the situation of asking this question).
Related
In Microsoft Visual Studio 2015 (v14.0) I have a solution that contains 3 projects.
Two of those projects are DLLs, and the other one is the executable.
The executable loads the DLLs at runtime and calls their functions and they exchange parameters; using Window's LoadLibrary, and GetProcAddress APIs.
In Release mode, when I set Runtime Library of my projects to Multi-threaded DLL everything works fine. This is Multi-threaded Debug DLL for Debug mode.
If I change to Multi-threaded for Release or Multi-threaded Debug for Debug I start getting Debug Assertion errors or Memory Access Violation errors and other kind of errors. (When I change it, I change it for all the projects in the solution.)
I need to use Multi-threaded option so that the executable won't need C++ runtime library on the target machine. How can I solve this issue?
It was because using /MT separates runtimes of entities (i.e. DLLs and executable), with each having their own runtime, hence their own heap, trying to allocate memory in one and freeing it in the other will end up in error. Because other modules were oblivious to the memory allocated.
On the other hand, with /MD all modules share the same runtime and as the runtime is aware of the memory allocated in one, it will be able to free it in the other. Because one instance of runtime manages the whole memory.
EDIT:
Thanks to dxiv comment.
The OpenSSL 1.0.2g package's INSTALL.W32 documentation has the following warning text:
One final comment about compiling applications linked to the OpenSSL library.
If you don't use the multithreaded DLL runtime library (/MD option) your
program will almost certainly crash because malloc gets confused -- the
OpenSSL DLLs are statically linked to one version, the application must
not use a different one.
I don't fully understand this or the repercussions of it. Are they saying that statically linking libeay32mt.lib is not supported?
We are experiencing random crashes in our app and the stack trace sometimes point to free calls in openssl functions, would that be the expected symptom that this warning refers to?
C and C++ memory management is implemented in the CRT (C Runtime Library). Each physical copy of the CRT mapped into a process (either statically compiled into a module, or referenced DLL) uses a distinct heap. Memory allocations and deallocations must be performed on the same heap, i.e. the same physical CRT copy (see Potential Errors Passing CRT Objects Across DLL Boundaries for more details).
In your specific scenario you need to make sure to do the following:
Dynamically link your application against the OpenSSL DLLs.
Dynamically link your application against the CRT.
Verify that both your application and the OpenSSL DLLs link against identical versions of the CRT (version and configuration).
While it is possible to statically link against OpenSSL, if you can ensure that your final binary contains a single CRT implementation and doesn't (directly or indirectly) dynamically link against the CRT as well. This is difficult to maintain, and breaks as soon as you link against a library, that doesn't provide a static library.
I had this bug today which turned out to be because I use a string allocated from inside my DLL after calling FreeLibrary().
This is a simple example reproducing the crash. This goes in DLL:
void dllFunc(char **output)
{
*output = strdup("Hello"); // strdup uses malloc
}
This is in the EXE that loads the DLL:
void exeFunc()
{
char *output;
dllFunc(&output);
std::string s1 = output; // This succeeds.
FreeLibrary(dll);
std::string s2 = output; // This crashes with access violation.
}
I read the documentation of FreeLibrary() but I couldn't find anything about memory becoming invalid after it's called.
Edit
I just realized that I had been using VS2008 toolchain for the DLL while using VS2010 toolchain for the EXE (I was using VS2010 as IDE for both, but you can select the toolchain from the project settings). Setting the toolchain to VS2010 for the DLL as well removed the crash.
If you choose static linking with the MSVCRT (C Runtime) library, you will get the behavior you describe. Same thing also happens if your EXE and DLL are dynamically linked to an MSVCRT DLL, but are using different versions. Or if they are matched to the same version, but one is using DEBUG and the other is using RETAIL. In other words, memory is only as good as the lifetime of the MSVCRTxxx.dll used to make the allocation. I just saw your update to your question - yes, mixing and matching the CRT between VS 2008 and 2010 is the exact reason for the crash.
If both your DLL and EXE are dynamically linked to the same version of the MSVCRT DLL, then you share the memory heap and you avoid the problem you are having.
The standard practice is this: If your exported DLL function returns anything that needs to be "freed" or "released" later, then standard practice is to provide an additional function exported out of the DLL to handle de-allocations.
You can configure both the EXE and DLL's C Runtime linkage from the Code-Generation page for the C/C++ project settings in your project.
Picture here: http://imgur.com/uld4KYF.png
This occurs because each Dll creates its own memory heap (which malloc and its C friends, along with new will use internally, generally via HeapAlloc), and when the Dll is freed, so is its heap.
Refer to this MSDN article for more Dll memory caveats. unless you are using a custom memory allocator, shared across all your binaries, you need to keep dynamically allocated memory within the module that created it (unless you can 100% guarantee that the object will not outlive its creator).
Is it possible to execute debug mode DLL with release mode EXE?
I am trying this scenario but EXE does not load the debug DLL and throws error "This application has failed to start...".
I know this is not good scenario to do but Due to certain requirements I have to make this work.
It can work if your dll interface has no dependencies on classes that may look different in debug and release.
e.g. std::string and std::vector in MSVC are not compatible in debug and release. (Fences ...)
So for example
std::string GetName();
will not work.
In additional new and delete should not be shifted because debug/release use different runtimes. But anyway you should always delete in the same context(dll/exe) as new.
Yes, this can work.
Your "application has failed to start" issue is most likely you copied a debug build of the DLL (built on your machine with Visual Studio), to a machine that did not have the DEBUG CRT installed. Usually copying over MSVCRTD(version).dll to the same directory as your program files solves this problem. I have a previous answer that covers some of this here.
Best bet is to always have all your binaries linked to the same dynamic MSVCRT DLL so they all share the same runtime.
Another easy workaround is to compile your DEBUG DLL to use the same flavor of the MSVCRT DLL (or statically link to the CRT). Somewhere in the VS project propery pages (code generation I think) is the dropdown for choosing the CRT. There's nothing wrong with linking the retail MSVCRT into a debug DLL - or statically linking.
The things to watch out for are when you have a different flavor of the debug C runtime linked to the different binaries. If you have the release MSVCRT dll linked for the EXE, but the debug MSCVRTD DLL for the DLL, that can cause problems in a few circumstances. This is because handles and memory blocks are tracked by two different instances of the CRT.
Examples:
If you allocate memory in the EXE, but free in in the DLL. And vice versa.
File handles opened with fopen() in the EXE, but used or closed in the EXE (and vice versa).
For any of the header files for the DLL's interface, having any sort of inline functions or methods implemented in the header file is an easy way to cause #1 or #2 to happen.
Sharing STL objects (std::string, std::list, std::vector) is a definite no-no for mixed CRT usage.
Due to how Microsoft implements the heap in their non-DLL versions of the runtime, returning a C++ object from a DLL can cause problems:
// dll.h
DLL_EXPORT std::string somefunc();
and:
// app.c - not part of DLL but in the main executable
void doit()
{
std::string str(somefunc());
}
The above code runs fine provided both the DLL and the EXE are built with the Multi-threaded DLL runtime library.
But if the DLL and EXE are built without the DLL runtime library (either the single or multi-threaded versions), the code above fails (with a debug runtime, the code aborts immediately due to the assertion _CrtIsValidHeapPointer(pUserData) failing; with a non-debug runtime the heap gets corrupted and the program eventually fails elsewhere).
Two questions:
Is there a way to solve this other then requiring that all code use the DLL runtime?
For people who distribute their libraries to third parties, how do you handle this? Do you not use C++ objects in your API? Do you require users of your library to use the DLL runtime? Something else?
Is there a way to solve this other then requiring that all code use the DLL runtime?
Not that I know of.
For people who distribute their libraries to third parties, how do you handle this? Do you not use C++ objects in your API? Do you require users of your library to use the DLL runtime? Something else?
In the past I distributed an SDK w/ dlls but it was COM based. With COM all the marshalling of parameters and IPC is done for you automatically. Users can also integrate in with any language that way.
Your code has two potential problems: you addressed the first one - CRT runtime. You have another problem here: the std::string could change among VC++ versions. In fact, it did change in the past.
The safe way to deal with is to export only C basic types. And exports both create and release functions from the DLL. Instead of export a std::string, export a pointer.
__declspec(export) void* createObject()
{
std::string* p = __impl_createObject();
return (void*)p;
}
__declspec(export) void releasePSTRING(void* pObj)
{
delete ((std::string*)(pObj));
}
There is a way to deal with this, but it's somewhat non-trivial. Like most of the rest of the library, std::string doesn't allocate memory directly with new -- instead, it uses an allocator (std::allocator<char>, by default).
You can provide your own allocator that uses your own heap allocation routines that are common to the DLL and the executable, such as by using HeapAlloc to obtain memory, and suballocate blocks from there.
If you have a DLL that you want to distribute and you don't want to bind your callers to a specific version to the C-Runtime, do any of the following:
I. Link the DLL to the static version of the C-Runtime library. From the Visual Studio Project Properties page, select the tab for Configuration Properties-> C/C++ -> Code Generation. These an option for selecting the "Runtime library". Select "Multithread" or "Multithreaded Debug" instead of the DLL versions. (Command line equilvalent is /MT or /MTd)
There are a couple of different drawbacks to this approach:
a. If Microsoft ever releases a security patch of the CRT, your shipped components may be vulnerable until your recompile and redist your binary.
b. Heap pointers allocated by "malloc" or "new" in the DLL can not be "free"d or "delete"d by the EXE or other binary. You'll crash otherwise. The same holds true for FILE handles created by fopen. You can't call fopen in teh DLL and expect the EXE to be able to fclose on it. Again, crash if you do. You will need to build the interface to your DLL to be resilient to all of these issues. For starters, a function that returns an instance to std::string is likely going to be an issue. Provide functions exported by your DLL to handle the releasing of resources as needed.
Other options:
II. Ship with no c-runtime depedency. This is a bit harder. You first have to remove all calls to the CRT out of your code, provide some stub functions to get the DLL to link, and specify "no default libraries" linking option. Can be done.
III. C++ classes can be exported cleanly from a DLL by using COM interface pointers. You'll still need to address the issues in 1a above, but ATL classes are a great way to get the overhead of COM out of the way.
The simple fact here is, Microsofts implementation aside, C++ is NOT an ABI. You cannot export C++ objects, on any platform, from a dynamic module, and expect them to work with a different compiler or language.
Exporting c++ classes from Dlls is a largely pointless excercise - because of the name mangling, and lack of support in c++ for dynamically loaded classes - the dlls have to be loaded statically - so you loose the biggest benefit of splitting a project into dlls - the ability to only load functionality as needed.