When an (unmanaged) Win32 program (or other DLL) calls an (unmanaged) Win32 DLL, my understanding is that there are a number of potential incompatibilities that could cause problems/crashes.
For instance, if program P calls DLL D, and P was compiled with Visual Studio 2013 to target Windows SDK version 8, and D was compiled to with Visual Studio 2017 to target Windows SDK version 10, my understanding is they will access different msvc runtime DLLs, which can cause problems. For instance, if D allocates memory with new and passes it back to P, and P tries to delete it, there is a crash because P and D are using two different runtimes, each maintaining their own heap.
Similarly, if P is built with run-time type information (RTTI) and D is not, my understanding is there can be crashing incompatibilities when, say, P allocates a class instance (to a type that each referenced in a common .h file) and passes it to D, because of differences in the virtual function table and/or in the padding between structure fields.
I'm trying to understand what other incompatibilities might exist. Ideally I'd like to find a complete list. Here are some possibilities that come to mind:
Struct Member Alignment (similar to the RTTI issue)
Use Multi-Byte Character Set vs. Use Unicode Character Set (could cause problems if string objects are passed/returned?)
Omit (vs. do not omit) frame pointers (or perhaps this does not apply because the function call mechanism across programs/DLLs is different from the internal function call mechanism this flag relates to?)
Enable (or disable) C++ Exceptions (exception handling works across a DLL call if they are both built with exceptions enabled and using the same runtime?)
Others?
EDIT
I asked this question as a general curiosity, but also because I was having random heap corruption using a DLL in my program, and I was trying to figure out what could possibly be the problem.
My issue turned out to be that I was linking against an outdated DLL import library. I'm not exactly sure why the program started without error, and why it basically ran okay but with intermittent heap corruption. I believe it could only start without error if the new DLL offered a superset of the functionality provided by the old DLL import library. And then I'm not sure if the mechanism is a mismatch in C/C++ runtime libraries, or perhaps a change in a function's parameters without a breaking change in function signature?
In any event, I guess linking against an outdated DLL import library can be added to the list, although it's a less innocent mistake than the others mentioned.
A very common mistake is using standard library stuff or other compiler implementation/version related features in dll-interfaces.
This will normally only work if both parties (dll and its user) use the same version of the compiler and even then there may be incompatibilities when one of them is build with DEBUG enabled and the other with NDEBUG.
Only use c-compatible types etc. in dll-interfaces.
Related
We have a executable compiled using Visual Studio 2008 version. Due to the 3rd party dependency we must compile this executable in visual studio 2008.
We also have another component which gets compiled in visual studio 2010. Now we need to get one COM component dll from this component (which is compiled in 2010 compiler version) accessed by the executable which is compiled using 2008 compiler version.
My question here is, would it work fine. Would there be conflicts in the runtime used by the executable (which is 2008 runtime lib) and runtime used by the COM component (which is using 2010 runtime).
We actually tried to load this COM dll in executable which actually worked fine. But I have concern that in later time due to multiple runtimes it may crash/fail.
Please let me know how the multiple runtimes would get handled here. Is it safe to load the different runtime in single executable. Would there be any conflicts in later part of execution due to different runtime available?
Anyway a solution we are looking to solve this problem to make the COM component as a OUT proc Server, which anyway will work. But that will involve a lot of work to do.
Please let me know.
Many Thanks
You should have no problem mixing COM objects that are linked with different runtime libraries, since the memory allocation and deallocation of each object will be done behind the DLL boundary.
You need to be careful that all your methods have proper COM signatures, i.e. all pointers should be COM pointers.
COM is designed for binary interop. By design the framework is implementation agnostic. The intent is that COM servers can be implemented in one language/runtime, and consumed by a COM client implemented with a different language/runtime.
There are absolutely no constraints over the languages and runtimes that are used by different parties.
This has been answered a few times in several contexts.
As long as you don't handle and/or pass around C runtime (CRT) data structures between modules, you're fine. If you do any of the following between modules that depend on different CRTs, you'll have trouble, and in this specific case, you're not implementing COM objects properly:
malloc memory in one module and realloc or free in another
fopen a FILE* in one module and fread, fwrite, fclose, etc. in another
setjmp in one module and longjmp in another
Note that there are things you can do:
Use memory malloced by another module, keeping the responsibility of reallocating and freeing on the originating module
Use some interface that interacts with files fopened by another module, keeping the responsibility of its use on the originating module
Don't use setjmp/longjmp across unrelated or losely coupled modules, define callbacks, aborting error codes, whatever, but don't rely on unwinding techniques, even if provided by the OS
You can see a pattern here. You can use resources from another module for as long as you delegate managing those resources to that module.
With COM, you shouldn't ever have this kind of trouble, everything should be encapsulated in objects through their implemented interfaces. Although you can pass malloced memory as top-level pointer arguments, you're only supposed to access that memory in the callee, never reallocate or free it. For inner pointers, you must use CoTaskMemAlloc and its cousins, as this is the common memory manager in COM. This applies to handling files (e.g. encapsulate them in IStream, IPipeByte, a IEnumByte or something similar), and don't unwind across COM calls.
How safe is linking MinGW excutable with a library that was compiled by Visual C++.
Something like what is explained here.
http://www.codesynthesis.com/~boris/blog/2011/12/09/oci-mingw/
TL; DR "... because OCI is a C library, we can take the “official” OCI import library for VC++, oci.lib, rename it to libclntsh.a, and we’ve got OCI for MinGW"
Is this an accident waiting to happen? What could go wrong?
It depends.
AFAIK, there is nothing to stop the glibc and msvcrt co-existing in the same process on Windows - the same global function search that occurs on Linux doesn't occur on windows (every dynamic import knows which DLL it comes from - functions are not merged in a single namespace).
However there may be problems with particular libraries. If the library specifies for example that "the function returns a pointer, which should be freed with free() when done", you need to free it with the correct free, i.e. the one corresponding to the malloc() it was allocated with. Likewise if the function states "parameter is a buffer which will be freed with free() by the function" then it must be allocated with the corresponding malloc(). Similar issues apply where realloc() may be used.
This issue also occurs for example when using DLLs compiled against differing versions of MSVCRT e.g. MSVCRT20.dll vs. MSVCRT40.dll.
That's why windows libraries always state how memory should be allocated. see for example CoTaskMemAlloc/CoTaskMemFree, LocalAlloc/LocalFree, HeapAlloc/HeapFree. Documentation may state "When no longer needed the buffere must be freed with CoTaskMemFree". Or they may supply their own free/alloc pairs e.g. "When no longer needed the returned buffer must be freed with SuperLibraryFreeBuffer" (which internally may simply delegate to the CRT free, but at least it will be the correct version of free).
This is because windows is and always has been a multi-language platform where libraries can be written in languages other than C. Today we may be used to the idea that Lisp, Pascal, etc are a layer on top of the C runtime, - most programmers probably assume that even when it isn't true as in the case of Pascal - but it wasn't always the way: Pascal was in common use on DEC computers for two years before C was invented, and the Windows heritage as everyone knows has a lot in common with DEC. Early versions of Windows were written largly in assembler and ... there is a reason for that "pascal" calling convention in the Windows 3 headers you know...
I have made a DLL with VC++ 2008 and when i use it in console application VC++ 6.0, there is an exception:
(msvcr90.dll): 0xc0000005: Access Violation
Access Violation in this case can mean so many things, and the msvcr90.dll reference can be very much misleading. If you pass invalid data to any of the MSVC standard library functions, the access violation will occur within msvcr90.dll and not in your code (when viewing the stack trace or looking at the exception info.
That said, there should not, in theory, be a problem using a VC9 DLL in VC++ 6, as the ABI has not changed and the PE format is the same. You might have problems if msvcrt9.dll is unsupported on your platform (for instance if you're running MSVC6 on Windows NT), but otherwise it means you need to review your code.
What I mean is: attach a debugger and look at what's happening beneath the scene!
One more note: when using different versions of the MSVC libraries dynamically, you MUST NOT allocate data in one library and free it in another as they are not guaranteed to be using the same heap and you can get memory corruption (and Access Violation errors) rather easily like this. This also means that if you're writing C++, you must not create an object and then pass it by return value to the calling app as that's what will happen beneath the scenes.
If you want to build a DLL with Visual C++ version X and use it in Visual C++ version Y, you have some options:
Build a DLL which exposes a pure C interface. You can use C++ inside the DLL, but the public interface must be pure C (so, for example, you can't throw exceptions crossing DLL boundaries).
Build a COM DLL (possibly with the help of tools like ATL).
Build a DLL using COM-like techniques, i.e. expose only abstract interfaces and factory functions from your DLL (this technique is explained in this article on CodeProject "HowTo: Export C++ classes from a DLL", in particular in the paragraph "C++ Mature Approach: Using an Abstract Interface").
It is also important to point out that the code which allocates memory and the code which frees memory must use the same allocator.
I use and Application compiled with the Visual C++ Compiler. It can load plugins in form of a .dll. It is rather unimportant what exactly it does, fact is:
This includes calling functions from the .dll that return a pointer to an object of the Applications API, etc.
My question is, what problems may appear when the Application calls a function from the .dll, retrieves a pointer from it and works with it. For example, something that comes into my mind, is the size of a pointer. Is it different in VC++ and G++? If yes, this would probably crash the Application?
I don't want to use the Visual Studio IDE (which is unfortunately the "preferred" way to use the Applications SDK). Can I configure G++ to compile like VC++?
PS: I use MINGW GNU G++
As long as both application and DLL are compiled on the same machine, and as long as they both only use the C ABI, you should be fine.
What you can certainly not do is share any sort of C++ construct. For example, you mustn't new[] an array in the main application and let the DLL delete[] it. That's because there is no fixed C++ ABI, and thus no way in which any given compiler knows how a different compiler implements C++ data structures. This is even true for different versions of MSVC++, which are not ABI-compatible.
All C++ language features are going to be entirely incompatible, I'm afraid. Everything from the name-mangling to memory allocation to the virtual-call mechanism are going to be completely different and not interoperable. The best you can hope for is a quick, merciful crash.
If your components only use extern "C" interfaces to talk to one another, you can probably make this work, although even there, you'll need to be careful. Both C++ runtimes will have startup and shutdown code, and there's no guarantee that whichever linker you use to assemble the application will know how to include this code for the other compiler. You pretty much must link g++-compiled code with g++, for example.
If you use C++ features with only one compiler, and use that compiler's linker, then it gets that much more likely to work.
This should be OK if you know what you are doing. But there's some things to watch out for:
I'm assuming the interface between EXE and DLL is a "C" interface or something COM like where the only C++ classes exposed are through pure-virutal interfaces. It gets messier if you are exporting a concrete class through a DLL.
32-bit vs. 64bit. The 32-bit app won't load a 64-bit DLL and vice-versa. Make sure they match.
Calling convention. __cdecl vs __stdcall. Often times Visual Studio apps are compiled with flags that assuming __stdcall as the default calling convention (or the function prototype explicitly says so). So make sure that the g++ compilers generates code that matches the calling type expected by the EXE. Otherwise, the exported function might run, but the stack can get trashed on return. If you debug through a crash like this, there's a good chance the cdecl vs stdcall convention was incorrectly specified. Easy to fix.
C-Runtimes will not likely be shared between the EXE and DLL, so don't mix and match. A pointer allocated with new or malloc in the EXE should not be released with delete or free in the DLL (and vice versa). Likewise, FILE handles returned by fopen() can not be shared between EXE and DLL. You'll likely crash if any of this happens.... which leads me to my next point....
C++ header files with inline code cause enough headaches and are the source of issues I called out in #3. You'll be OK if the interface between DLL And EXE is a pure "C" interface.
Name mangling issues. If you run into issues where the function name exported doesn't match because of name mangling or because of a leading underscore, you can fix that up in a .DEF file. At least that's what I've done in the past with Visual Studio. Not sure if the equivalent exists in g++/MinGW. Example below. Learn to use "dumpbin.exe /exports" to you can validate your DLL is exporting function with the right name. Using extern "C" will also help fix this as well.
EXPORTS
FooBar=_Foobar#12
BlahBlah=??BlahBlah##QAE#XZ #236 NONAME
Those are the issues that I know of. I can't tell you much more since you didn't explain the interface between the DLL and EXE.
The size of a pointer won't vary; that is dependent on the platform and module bitness and not the compiler (32-bit vs 64-bit and so on).
What can vary is the size of basically everything else, and what will vary are templates.
Padding and alignment of structs tends to be compiler-dependent, and often settings-within-compiler dependent. There are so loose rules, like pointers typically being on a platform-bitness-boundary and bools having 3 bytes after them, but it's up to the compiler how to handle that.
Templates, particularly from the STL (which is different for each compiler) may have different members, sizes, padding, and mostly anything. The only standard part is the API, the backend is left to the STL implementation (there are some rules, but compilers can still compile templates differently). Passing templates between modules from one build is bad enough, but between different compilers it can often be fatal.
Things which aren't standardized (name mangling) or are highly specific by necessity (memory allocation) will also be incompatible. You can get around both of those issues by only destroying from the library that creates (good practice anyway) and using STL objects that take a deleter, for allocation, and exporting using undecorated names and/or the C style (extern "C") for exported methods.
I also seem to remember a catch with how the compilers handle virtual destructors in the vtable, with some small difference.
If you can manage to only pass references of your own objects, avoid externally visible templates entirely, work primarily with pointers and exported or virtual methods, you can avoid a vast majority of the issues (COM does precisely this, for compatibility with most compilers and languages). It can be a pain to write, but if you need that compatibility, it is possible.
To alleviate some, but not all, of the issues, using an alternate to the STL (like Qt's core library) will remove that particular problem. While throwing Qt into any old project is a hideous waste and will cause more bloat than the "boost ALL THE THINGS!!!" philosophy, it can be useful for decoupling the library and the compiler to a greater extent than using a stock STL can.
You can't pass C runtime objects between them. For example you can not open a FILE buffer in one and pass it to be used in the other. You can't free memory allocated on the other side.
The main problems are the function signatures and way parameters are passed to library code. I've had great difficulty getting VC++ dll's to work in gnu based compilers in the past. This was way back when VC++ always cost money and mingw was the free solution.
My experience was with DirectX API's. Slowly a subset got it's binaries modified by enthusiasts but it was never as up-to-date or reliable so after evaluating it I switched to a proper cross platform API, that was SDL.
This wikipedia article describes the different ways libraries can be compiled and linked. It is rather more in depth than I am able to summarise here.
I'd like to build certain extensions of my game into a DLL so that I can recompile the extensions and replace the DLL without rebuilding the entire game. How do I go about doing that?
I read somewhere that each new version of the DLL would have to be "binary compatible" with the old one. Are DLLs compiled with visual studio binary compatible by default as long as I don't change the function names?
You have to do more than just keep the function names the same. What you have to ensure is called Application Binary Interface, or ABI, compatibility.
There's lots of stuff this entails beyond just symbol names. For example, you have do things like to keep data structures that are passed across the DLL boundary the same size, and the compiler has to place any class members defined in your module but accessed outside of it in the same relative location. A lot of it is up to the compiler's ABI. Microsoft compilers have a bad habit of not being ABI compatible across versions, while some other compilers work a little harder at being compatible. But remember that it requires more than just using the same compiler.
You'll probably need to read up on maintaining ABI compatibility in your environment because it probably involves more details than could be listed here and there are lots of ways to go wrong. Here's one short article on it. There may be better ones specifically for MSVC compilers.
In case you export classes from your DLL you will break the compatibility if you change a version of compiler. In order to solve this problem do this.
Make class interfaces like :
class EXPORT_IMPORT_MACROS IMyApi{
public:
virtual void doSomething( IOtherApi* api ) = 0;
virtual void doSomething2( IOtherApi2* api ) = 0;
};
and export them from your dll. It will give you a header files which are independent from dll implementation and also exportable that means that it will not generate any problem with different versions of runtime library. Also you could take a look to COM technology.
One way to do it is recompiling the DLLs. The DLLs are compatible provided that the version of compiler (Visual Studio) doesn't change. Otherwise you will have to check the manual for compatibility between Visual Studio versions.
Another way of doing this is by loading the dll dynamically by using LoadLibrary function and after that using GetProcAddress function to retrieve the address of the function you want to call.
If you use the latter you can use any compiler as long as you export the functions correctly using dllexport and import them with dllimport. More details here.