The OpenSSL 1.0.2g package's INSTALL.W32 documentation has the following warning text:
One final comment about compiling applications linked to the OpenSSL library.
If you don't use the multithreaded DLL runtime library (/MD option) your
program will almost certainly crash because malloc gets confused -- the
OpenSSL DLLs are statically linked to one version, the application must
not use a different one.
I don't fully understand this or the repercussions of it. Are they saying that statically linking libeay32mt.lib is not supported?
We are experiencing random crashes in our app and the stack trace sometimes point to free calls in openssl functions, would that be the expected symptom that this warning refers to?
C and C++ memory management is implemented in the CRT (C Runtime Library). Each physical copy of the CRT mapped into a process (either statically compiled into a module, or referenced DLL) uses a distinct heap. Memory allocations and deallocations must be performed on the same heap, i.e. the same physical CRT copy (see Potential Errors Passing CRT Objects Across DLL Boundaries for more details).
In your specific scenario you need to make sure to do the following:
Dynamically link your application against the OpenSSL DLLs.
Dynamically link your application against the CRT.
Verify that both your application and the OpenSSL DLLs link against identical versions of the CRT (version and configuration).
While it is possible to statically link against OpenSSL, if you can ensure that your final binary contains a single CRT implementation and doesn't (directly or indirectly) dynamically link against the CRT as well. This is difficult to maintain, and breaks as soon as you link against a library, that doesn't provide a static library.
Related
Is it possible to execute debug mode DLL with release mode EXE?
I am trying this scenario but EXE does not load the debug DLL and throws error "This application has failed to start...".
I know this is not good scenario to do but Due to certain requirements I have to make this work.
It can work if your dll interface has no dependencies on classes that may look different in debug and release.
e.g. std::string and std::vector in MSVC are not compatible in debug and release. (Fences ...)
So for example
std::string GetName();
will not work.
In additional new and delete should not be shifted because debug/release use different runtimes. But anyway you should always delete in the same context(dll/exe) as new.
Yes, this can work.
Your "application has failed to start" issue is most likely you copied a debug build of the DLL (built on your machine with Visual Studio), to a machine that did not have the DEBUG CRT installed. Usually copying over MSVCRTD(version).dll to the same directory as your program files solves this problem. I have a previous answer that covers some of this here.
Best bet is to always have all your binaries linked to the same dynamic MSVCRT DLL so they all share the same runtime.
Another easy workaround is to compile your DEBUG DLL to use the same flavor of the MSVCRT DLL (or statically link to the CRT). Somewhere in the VS project propery pages (code generation I think) is the dropdown for choosing the CRT. There's nothing wrong with linking the retail MSVCRT into a debug DLL - or statically linking.
The things to watch out for are when you have a different flavor of the debug C runtime linked to the different binaries. If you have the release MSVCRT dll linked for the EXE, but the debug MSCVRTD DLL for the DLL, that can cause problems in a few circumstances. This is because handles and memory blocks are tracked by two different instances of the CRT.
Examples:
If you allocate memory in the EXE, but free in in the DLL. And vice versa.
File handles opened with fopen() in the EXE, but used or closed in the EXE (and vice versa).
For any of the header files for the DLL's interface, having any sort of inline functions or methods implemented in the header file is an easy way to cause #1 or #2 to happen.
Sharing STL objects (std::string, std::list, std::vector) is a definite no-no for mixed CRT usage.
I've been having a lot of conceptual issues with Microsoft's CRT. For any project you have to compile all required libraries to link against the same version of the CRT.
The first problem is when your project statically links against the CRT (/MT). Then all the dependant libraries must also link their own CRT statically. So each library has its own version of - for example - malloc(). If you compiled one of the libraries last year on system A, that CRT version may be different than the one you're currently using on another system B with service pack 3+. So if you're freeing objects allocated by the library you may run into problems.
So it seems dynamically linked CRT is the way to go (/MD). With dlls all the libraries would get the current implementation of the CRT on the system. Except that with Microsoft's Side by Side mechanism that's not what happens. Instead you get the CRT version that's stamped on the library you compiled and that version of the DLL is supplied to that library. So exactly the same problem I described before can occur. You compile a library on system A a year ago against one CRT. A year later there's a new version with upgrade. Your main program gets the DLL with one version of the CRT, the library gets the DLL with another version of CRT, same problem can occur.
So what do you do? I realize cross library memory allocation is frowned upon. But you can ignore the malloc example and come up with another one. Do you have every developer recompile every dependant library on their machine to make sure everything does use the same CRT? Then for the release you recompile every library again?
How does this work on Linux? That's my main interest. Is there a CRT supplied with GCC or the Linux system itself comes with CRT libraries? I've never seen the CRT linked explicitly in Makefils.
On Linux, what CRT do dynamic libraries link against? The most current one on the machine, or is it more "side by side" mechanism.
On the Linux side I think there are two basic parts of the standard library that are at issue: We have the C-runtime part, which should pretty much be ABI compatible forever. Effectively whichever version links at final link time should be fine, and you can redistribute any needed shared library with your binary if it's an older version needed for compatibility. Usually the libraries just sit side-by-side on *NIX systems.
Secondly, you have the C++ libraries. These are pretty much guaranteed to not be ABI compatible in any way, so you must rebuild every single component of a final binary against the same version of the C++ library. There's just no way around it unfortunately because otherwise you could wind up with a variety of mismatches. This is why many open source libraries don't even bother with premade library binaries: Everyone needs to build their own copy to make sure that it will properly link into their final application code.
Due to how Microsoft implements the heap in their non-DLL versions of the runtime, returning a C++ object from a DLL can cause problems:
// dll.h
DLL_EXPORT std::string somefunc();
and:
// app.c - not part of DLL but in the main executable
void doit()
{
std::string str(somefunc());
}
The above code runs fine provided both the DLL and the EXE are built with the Multi-threaded DLL runtime library.
But if the DLL and EXE are built without the DLL runtime library (either the single or multi-threaded versions), the code above fails (with a debug runtime, the code aborts immediately due to the assertion _CrtIsValidHeapPointer(pUserData) failing; with a non-debug runtime the heap gets corrupted and the program eventually fails elsewhere).
Two questions:
Is there a way to solve this other then requiring that all code use the DLL runtime?
For people who distribute their libraries to third parties, how do you handle this? Do you not use C++ objects in your API? Do you require users of your library to use the DLL runtime? Something else?
Is there a way to solve this other then requiring that all code use the DLL runtime?
Not that I know of.
For people who distribute their libraries to third parties, how do you handle this? Do you not use C++ objects in your API? Do you require users of your library to use the DLL runtime? Something else?
In the past I distributed an SDK w/ dlls but it was COM based. With COM all the marshalling of parameters and IPC is done for you automatically. Users can also integrate in with any language that way.
Your code has two potential problems: you addressed the first one - CRT runtime. You have another problem here: the std::string could change among VC++ versions. In fact, it did change in the past.
The safe way to deal with is to export only C basic types. And exports both create and release functions from the DLL. Instead of export a std::string, export a pointer.
__declspec(export) void* createObject()
{
std::string* p = __impl_createObject();
return (void*)p;
}
__declspec(export) void releasePSTRING(void* pObj)
{
delete ((std::string*)(pObj));
}
There is a way to deal with this, but it's somewhat non-trivial. Like most of the rest of the library, std::string doesn't allocate memory directly with new -- instead, it uses an allocator (std::allocator<char>, by default).
You can provide your own allocator that uses your own heap allocation routines that are common to the DLL and the executable, such as by using HeapAlloc to obtain memory, and suballocate blocks from there.
If you have a DLL that you want to distribute and you don't want to bind your callers to a specific version to the C-Runtime, do any of the following:
I. Link the DLL to the static version of the C-Runtime library. From the Visual Studio Project Properties page, select the tab for Configuration Properties-> C/C++ -> Code Generation. These an option for selecting the "Runtime library". Select "Multithread" or "Multithreaded Debug" instead of the DLL versions. (Command line equilvalent is /MT or /MTd)
There are a couple of different drawbacks to this approach:
a. If Microsoft ever releases a security patch of the CRT, your shipped components may be vulnerable until your recompile and redist your binary.
b. Heap pointers allocated by "malloc" or "new" in the DLL can not be "free"d or "delete"d by the EXE or other binary. You'll crash otherwise. The same holds true for FILE handles created by fopen. You can't call fopen in teh DLL and expect the EXE to be able to fclose on it. Again, crash if you do. You will need to build the interface to your DLL to be resilient to all of these issues. For starters, a function that returns an instance to std::string is likely going to be an issue. Provide functions exported by your DLL to handle the releasing of resources as needed.
Other options:
II. Ship with no c-runtime depedency. This is a bit harder. You first have to remove all calls to the CRT out of your code, provide some stub functions to get the DLL to link, and specify "no default libraries" linking option. Can be done.
III. C++ classes can be exported cleanly from a DLL by using COM interface pointers. You'll still need to address the issues in 1a above, but ATL classes are a great way to get the overhead of COM out of the way.
The simple fact here is, Microsofts implementation aside, C++ is NOT an ABI. You cannot export C++ objects, on any platform, from a dynamic module, and expect them to work with a different compiler or language.
Exporting c++ classes from Dlls is a largely pointless excercise - because of the name mangling, and lack of support in c++ for dynamically loaded classes - the dlls have to be loaded statically - so you loose the biggest benefit of splitting a project into dlls - the ability to only load functionality as needed.
I have an EXE file using a DLL file which is using another DLL file. This situation has arisen:
In DLL file 1:
class abc
{
static bool FindSubFolders(const std::string & sFolderToCheck,
std::vector< std::string > & vecSubFoldersFound);
}
In DLL file 2:
void aFunction()
{
std::vector<std::string> folders;
std::string sLocation;
...
abc::FindSubFolders(sLocation, folders)
}
In release mode, everything works fine. But in debug mode, I come up with an assertion failure in the destructor of one of the std::strings in the folders vector (when folders goes out of scope at the end of aFunction):
dbgheap.c : line 1274
/*
* If this ASSERT fails, a bad pointer has been passed in. It may be
* totally bogus, or it may have been allocated from another heap.
* The pointer MUST come from the 'local' heap.
*/
_ASSERTE(_CrtIsValidHeapPointer(pUserData));
I assume this is because the memory has been allocated on DLL file 1's heap, but is being freed in DLL file 2.
The comment in dbgheap.c seems pretty insistent that this is a problem.
Why is this such a problem, when it seems to work fine if I just ignore it? Is there a non-assertion-failing way of doing this?
As sean has already said, the release build simply ignores that delete statement, so the best you can hope for is a memory leak.
If you have control over how both DLL files are compiled, make sure to use the Multi-threaded Debug DLL (/MDd) or Multi-threaded DLL (/MD) settings for the runtime library. That way, both DLL files will use the same runtime system and share the same heap.
The downside is that you need to install the runtime system together with your application (Microsoft offers an installer for that). It will work fine on your development machine since Visual Studio installs that runtime system too, but on a freshly installed machine it will report missing DLL files.
Most likely, the release build has the same problem, but release builds don't assert. They just ignore the problem. You might never see an issue. Or you might see data corruption. Or you might see a crash. Maybe only your users will experience bugs that you are simply not able to reproduce.
Don't ignore CRT assertions.
You should always use the appropriate deallocator (the one that matches the allocator used to begin with). If you are using static CRT libraries in your DLL files, the DLL files are using different heaps. You can not deallocate memory across heaps. Allocate and deallocate a block of memory using the same heap.
If you are using shared CRT libraries in your DLL files, then they should be using the same heap and you can allocate in one DLL file and deallocate in another.
As other says, the problem can be solved by making sure that the CRT is shared between the two modules. But there are common scenarios where this contract is hard to enforce.
The reason is that making sure to link against a shared CRT will not work if the EXE and DLL do not link against the same CRT version (as in 6.0, 7.0, 8.0). For example if you take a DLL that has been built in VC6.0 and try to use it with an EXE build in VS2010 you will get the same issue as before. The two versions of CRT will be loaded in your process and will each use their own heap for allocation, regardless if your EXE and DLL use 'shared' CRT, they will not be the same.
A better practice for an API would be to make sure that objects allocated in one side are also destroyed on the same side. I know this sounds ugly but it is the only way to ensure that a DLL remains binary compatible.
This is only a problem if the application or one (or more) of the DLL files is linked against the static version of the standard library. This was solved about a decade ago by MS releasing the shared library version of the standard library. This is because each version of the standard library will build it own internal heap and thus with multiple heaps you must release the memory to the correct heap. By using the shared version of the standard library they all use the same heap.
It is standard practice nowadays for the application and all DLL files should be built to use the dynamic version of the standard library.
The only caveat to the above is if you create your own heap and allocate memory from this heap. But this is a very specialized procedure only done in rare situations (and if you understand enough to use it then you will not be in the situation of asking this question).
I would like some information on the runtime libraries for Visual Studio 2008. Most specifically when should I consider the DLL versions and when should I consider the Static versions.
The Visual Studio documentation delineates the technical differences in terms of DLL dependencies and linked libraries. But I'm left wondering why I should want to use one over the other. More important, why should I want to use the multi-threaded DLL runtime when this obviously forces my application into a DLL dependency, whereas the static runtime has no such requirement on my application user machine.
Linking dynamically to the runtime libraries complicates deployment slightly due to the DLL dependency, but also allows your application to take advantage of updates (bug fixes or more likely performance improvements) to the MS runtime libraries without being recompiled.
Statically linking simplifies deployment, but means that your application must be recompiled against newer versions of the runtime in order to use them.
Larry Osterman feels that you should always use the multi-threaded DLL for application programming. To summarize:
Your app will be smaller
Your app will load faster
Your app will support multiple threads without changing the library dependency
Your app can be split into multiple DLLs more easily (since there will only be one instance of the runtime library loaded)
Your app will automagically stay up to date with security fixes shipped by Microsoft
Please read his whole blog post for full details.
On the downside, you need to redistribute the runtime library, but that's commonly done and you can find documentation on how to include it in your installer.
Dynamically linking the runtime library can give you faster program start up times and smaller system memory usage since the dll can be shared between processes and won't need to be loaded again if it's already used by another process.
I think that main difference is how exceptions will be processed. Microsoft doesn't recommend to link statically to the CRT in a DLL unless the consequences of this are specifically desired and understood:
For example, if you call _set_se_translator in an executable that loads the DLL linked to its own static CRT, any hardware exceptions generated by the code in the DLL will not be caught by the translator, but hardware exceptions generated by code in the main executable will be caught.