I'd like to build certain extensions of my game into a DLL so that I can recompile the extensions and replace the DLL without rebuilding the entire game. How do I go about doing that?
I read somewhere that each new version of the DLL would have to be "binary compatible" with the old one. Are DLLs compiled with visual studio binary compatible by default as long as I don't change the function names?
You have to do more than just keep the function names the same. What you have to ensure is called Application Binary Interface, or ABI, compatibility.
There's lots of stuff this entails beyond just symbol names. For example, you have do things like to keep data structures that are passed across the DLL boundary the same size, and the compiler has to place any class members defined in your module but accessed outside of it in the same relative location. A lot of it is up to the compiler's ABI. Microsoft compilers have a bad habit of not being ABI compatible across versions, while some other compilers work a little harder at being compatible. But remember that it requires more than just using the same compiler.
You'll probably need to read up on maintaining ABI compatibility in your environment because it probably involves more details than could be listed here and there are lots of ways to go wrong. Here's one short article on it. There may be better ones specifically for MSVC compilers.
In case you export classes from your DLL you will break the compatibility if you change a version of compiler. In order to solve this problem do this.
Make class interfaces like :
class EXPORT_IMPORT_MACROS IMyApi{
public:
virtual void doSomething( IOtherApi* api ) = 0;
virtual void doSomething2( IOtherApi2* api ) = 0;
};
and export them from your dll. It will give you a header files which are independent from dll implementation and also exportable that means that it will not generate any problem with different versions of runtime library. Also you could take a look to COM technology.
One way to do it is recompiling the DLLs. The DLLs are compatible provided that the version of compiler (Visual Studio) doesn't change. Otherwise you will have to check the manual for compatibility between Visual Studio versions.
Another way of doing this is by loading the dll dynamically by using LoadLibrary function and after that using GetProcAddress function to retrieve the address of the function you want to call.
If you use the latter you can use any compiler as long as you export the functions correctly using dllexport and import them with dllimport. More details here.
Related
When an (unmanaged) Win32 program (or other DLL) calls an (unmanaged) Win32 DLL, my understanding is that there are a number of potential incompatibilities that could cause problems/crashes.
For instance, if program P calls DLL D, and P was compiled with Visual Studio 2013 to target Windows SDK version 8, and D was compiled to with Visual Studio 2017 to target Windows SDK version 10, my understanding is they will access different msvc runtime DLLs, which can cause problems. For instance, if D allocates memory with new and passes it back to P, and P tries to delete it, there is a crash because P and D are using two different runtimes, each maintaining their own heap.
Similarly, if P is built with run-time type information (RTTI) and D is not, my understanding is there can be crashing incompatibilities when, say, P allocates a class instance (to a type that each referenced in a common .h file) and passes it to D, because of differences in the virtual function table and/or in the padding between structure fields.
I'm trying to understand what other incompatibilities might exist. Ideally I'd like to find a complete list. Here are some possibilities that come to mind:
Struct Member Alignment (similar to the RTTI issue)
Use Multi-Byte Character Set vs. Use Unicode Character Set (could cause problems if string objects are passed/returned?)
Omit (vs. do not omit) frame pointers (or perhaps this does not apply because the function call mechanism across programs/DLLs is different from the internal function call mechanism this flag relates to?)
Enable (or disable) C++ Exceptions (exception handling works across a DLL call if they are both built with exceptions enabled and using the same runtime?)
Others?
EDIT
I asked this question as a general curiosity, but also because I was having random heap corruption using a DLL in my program, and I was trying to figure out what could possibly be the problem.
My issue turned out to be that I was linking against an outdated DLL import library. I'm not exactly sure why the program started without error, and why it basically ran okay but with intermittent heap corruption. I believe it could only start without error if the new DLL offered a superset of the functionality provided by the old DLL import library. And then I'm not sure if the mechanism is a mismatch in C/C++ runtime libraries, or perhaps a change in a function's parameters without a breaking change in function signature?
In any event, I guess linking against an outdated DLL import library can be added to the list, although it's a less innocent mistake than the others mentioned.
A very common mistake is using standard library stuff or other compiler implementation/version related features in dll-interfaces.
This will normally only work if both parties (dll and its user) use the same version of the compiler and even then there may be incompatibilities when one of them is build with DEBUG enabled and the other with NDEBUG.
Only use c-compatible types etc. in dll-interfaces.
Try to don't judge me really hard for this question.
Will I have problems if I try to use a DLL (written in C++) in a Visual Studio project that was compiled with the MinGW toolchain (g++)?
If the answer is yes, could someone explain me why?
As far as code execution goes, if it targets Windows, then it should run.
For interoperability; it depends on the API exposed from the dll.
If it is a "C" style API then you should be fine.
If it is a C++ API with classes and STL use, then you may run into trouble using different compilers (even different settings in the compilers). C++ doesn't have a standard ABI.
As noted in the comments and probably worth touching on here; the "C" style API is more than just the function calls, compatibility includes (but not limited) to things such as;
Memory allocation; general rule is "deallocation must be done by the same code as the allocation" - i.e. in the dll such that the matching allocation/deallocation is used (e.g. new and delete, malloc and free, CustomAlloc and CustomDealloc)
Data alignment and packing, not all compilers have the same defaults
Calling conventions, again, not all compilers have the same defaults, when in doubt, be explicit on what that convention should be (e.g. __stdcall)
As for the C++ ABI, in addition to all of the above, incompatibilities include;
Name mangling
STL class and container implementations
Separate and different implementations of new and delete, with custom memory pooling etc.
Exception mechanisms, e.g. generation, and stack unwinding
These are not comprehensive lists, but they go some way to motivating the general advice that when possible, use the same toolchains and same compiler/linker settings.
The problem is not about DLL. It's about how you describe such DLL to your compiler (ie. header files and exports).
Those descriptions affect how your code expect on data layout (e.g. different compiler may have different default alignment and padding), or even different code path (e.g. resolve to different inline functions and macro, such as in STL case).
Then each compiler may have different name mangling scheme, you may need a bit hack to glue the codes.
With very special care on the above issues you should be fine using a DLL from different compiler.
No, because it is all one standard. And in fact your C++ application compiled with gcc calls system functions from OS dll files. So no matter if you call them manually or automatically, it is all the same.
Can a std::string be passed by value across DLL boundries between DLLs built with different version of Visual Studio?
No, because templated code is generated separately per module.
So, when your EXE instantiates a std::string and passes it to the DLL, the DLL will begin using a completely different implementation on it. The result is a total mess, but it often sorta almost works because implementations are very similar, or the mess is hard to detect because it's some kind of subtle heap corruption.
Even if they're both built with the same version of VS, it's very precarious / fragile, and I would not recommend it. Either use a C-style interface between modules (for example, COM), or just don't use a DLL.
More detailed explanation here: Creating c++ DLL without static methods
and here: How can I call a function of a C++ DLL that accepts a parameter of type stringstream from C#?
In general, you can not mix binary code built with different compilers, which includes different versions of the same compiler (and can even include the same compiler invoked with different commandline options), so the answer to what you are trying to do is a clear "No".
The reason is that different compilers might provide different implementations of std::string. For example, one implementation could have a fixed, static buffer, while another version doesn't, which already leads to different object sizes. There is a bunch of other things that can make the interfaces incompatible, like the underlying allocator, the internal representation. Some things will already fail to link, due to name mangling or different private APIs, which both protect you from doing something wrong.
Some notes:
Even if you didn't pass the object by value but by reference, the called code could have a different idea of what this objects looks like.
It also doesn't matter that the type is supplied by the compiler, even if you defined the class yourself and compiled two DLLs with different versions of the class definition you would have problems. Also, if you change your standard library implementation, you make your binaries incompatible, too.
It doesn't even matter that the other code is in a DLL, it also applies to code in the same executable or DLL, although headers and automatic recompilation on change make this case very unlikely.
Specifically for MS Windows, you also have a debug heap and a release heap, and memory allocated in one must not be returned to the other. For that reason, you often have two DLL, one with a 'd' suffix (the debug version) and one without. This is a case where the compiler settings already affect compatibility, but you can get around this using a parallel approach of providing two versions of your DLL, too.
To some degree, similar problems occur for C code, too, where compilers have to agree on e.g. struct layout and calling conventions. Due to greater age and lower complexity, different C compilers are effectively compatible though. This is also accepted as a necessary feature in C as opposed to C++.
I use and Application compiled with the Visual C++ Compiler. It can load plugins in form of a .dll. It is rather unimportant what exactly it does, fact is:
This includes calling functions from the .dll that return a pointer to an object of the Applications API, etc.
My question is, what problems may appear when the Application calls a function from the .dll, retrieves a pointer from it and works with it. For example, something that comes into my mind, is the size of a pointer. Is it different in VC++ and G++? If yes, this would probably crash the Application?
I don't want to use the Visual Studio IDE (which is unfortunately the "preferred" way to use the Applications SDK). Can I configure G++ to compile like VC++?
PS: I use MINGW GNU G++
As long as both application and DLL are compiled on the same machine, and as long as they both only use the C ABI, you should be fine.
What you can certainly not do is share any sort of C++ construct. For example, you mustn't new[] an array in the main application and let the DLL delete[] it. That's because there is no fixed C++ ABI, and thus no way in which any given compiler knows how a different compiler implements C++ data structures. This is even true for different versions of MSVC++, which are not ABI-compatible.
All C++ language features are going to be entirely incompatible, I'm afraid. Everything from the name-mangling to memory allocation to the virtual-call mechanism are going to be completely different and not interoperable. The best you can hope for is a quick, merciful crash.
If your components only use extern "C" interfaces to talk to one another, you can probably make this work, although even there, you'll need to be careful. Both C++ runtimes will have startup and shutdown code, and there's no guarantee that whichever linker you use to assemble the application will know how to include this code for the other compiler. You pretty much must link g++-compiled code with g++, for example.
If you use C++ features with only one compiler, and use that compiler's linker, then it gets that much more likely to work.
This should be OK if you know what you are doing. But there's some things to watch out for:
I'm assuming the interface between EXE and DLL is a "C" interface or something COM like where the only C++ classes exposed are through pure-virutal interfaces. It gets messier if you are exporting a concrete class through a DLL.
32-bit vs. 64bit. The 32-bit app won't load a 64-bit DLL and vice-versa. Make sure they match.
Calling convention. __cdecl vs __stdcall. Often times Visual Studio apps are compiled with flags that assuming __stdcall as the default calling convention (or the function prototype explicitly says so). So make sure that the g++ compilers generates code that matches the calling type expected by the EXE. Otherwise, the exported function might run, but the stack can get trashed on return. If you debug through a crash like this, there's a good chance the cdecl vs stdcall convention was incorrectly specified. Easy to fix.
C-Runtimes will not likely be shared between the EXE and DLL, so don't mix and match. A pointer allocated with new or malloc in the EXE should not be released with delete or free in the DLL (and vice versa). Likewise, FILE handles returned by fopen() can not be shared between EXE and DLL. You'll likely crash if any of this happens.... which leads me to my next point....
C++ header files with inline code cause enough headaches and are the source of issues I called out in #3. You'll be OK if the interface between DLL And EXE is a pure "C" interface.
Name mangling issues. If you run into issues where the function name exported doesn't match because of name mangling or because of a leading underscore, you can fix that up in a .DEF file. At least that's what I've done in the past with Visual Studio. Not sure if the equivalent exists in g++/MinGW. Example below. Learn to use "dumpbin.exe /exports" to you can validate your DLL is exporting function with the right name. Using extern "C" will also help fix this as well.
EXPORTS
FooBar=_Foobar#12
BlahBlah=??BlahBlah##QAE#XZ #236 NONAME
Those are the issues that I know of. I can't tell you much more since you didn't explain the interface between the DLL and EXE.
The size of a pointer won't vary; that is dependent on the platform and module bitness and not the compiler (32-bit vs 64-bit and so on).
What can vary is the size of basically everything else, and what will vary are templates.
Padding and alignment of structs tends to be compiler-dependent, and often settings-within-compiler dependent. There are so loose rules, like pointers typically being on a platform-bitness-boundary and bools having 3 bytes after them, but it's up to the compiler how to handle that.
Templates, particularly from the STL (which is different for each compiler) may have different members, sizes, padding, and mostly anything. The only standard part is the API, the backend is left to the STL implementation (there are some rules, but compilers can still compile templates differently). Passing templates between modules from one build is bad enough, but between different compilers it can often be fatal.
Things which aren't standardized (name mangling) or are highly specific by necessity (memory allocation) will also be incompatible. You can get around both of those issues by only destroying from the library that creates (good practice anyway) and using STL objects that take a deleter, for allocation, and exporting using undecorated names and/or the C style (extern "C") for exported methods.
I also seem to remember a catch with how the compilers handle virtual destructors in the vtable, with some small difference.
If you can manage to only pass references of your own objects, avoid externally visible templates entirely, work primarily with pointers and exported or virtual methods, you can avoid a vast majority of the issues (COM does precisely this, for compatibility with most compilers and languages). It can be a pain to write, but if you need that compatibility, it is possible.
To alleviate some, but not all, of the issues, using an alternate to the STL (like Qt's core library) will remove that particular problem. While throwing Qt into any old project is a hideous waste and will cause more bloat than the "boost ALL THE THINGS!!!" philosophy, it can be useful for decoupling the library and the compiler to a greater extent than using a stock STL can.
You can't pass C runtime objects between them. For example you can not open a FILE buffer in one and pass it to be used in the other. You can't free memory allocated on the other side.
The main problems are the function signatures and way parameters are passed to library code. I've had great difficulty getting VC++ dll's to work in gnu based compilers in the past. This was way back when VC++ always cost money and mingw was the free solution.
My experience was with DirectX API's. Slowly a subset got it's binaries modified by enthusiasts but it was never as up-to-date or reliable so after evaluating it I switched to a proper cross platform API, that was SDL.
This wikipedia article describes the different ways libraries can be compiled and linked. It is rather more in depth than I am able to summarise here.
I have a question about shared libraries/.dlls and class importing/exporting. From what I have understood after some research the two ways to do it is as follows:
Just using __declspec(dllexport/dllimport) before the class and praying that the different compiler versions mangle the names in the same way, and accept the fact that your .dll will not be useable by different compilers.
Use pure virtual classes in the dll as interfaces. Then all the classes that would need exporting would inherit from them and should implement the virtual functions. What the .dll would export in this case is just "factory" Construct/Destruct functions which would create and release the objects.
These are the only two ways I know. The first is a no-no since it offers 0 portability. The second, though convenient and a good programming design for a .dll that accomplishes a purpose starts to be annoying when you realize that you need to have a different constructor function for every different constructor, can only use POD types as parameters and you lose many advantages of C++ classes such as overloaded functions and default function arguments.
For a .dll that is supposed to offer a library to the user, a collection of classes for example even the second way becomes really inconvenient. So I am wondering what is the solution here? Just compile a different .dll with the first way for each major compiler? How do the shared libraries version of big libraries work? For example wxWidgets does offer a .dll version too. How do they achieve normal usage of classes by the end user of the .dll avoiding the interface solution?
Solution with having separate DLL for each compiler version works. At the same time it is still not an official feature. You will never know if next service pack will break the compatibility or not, nobody will give you exact list of compiler/linker keys that will maintain/break the compatibility, etc. I heard reliable rumors that Windows8 finally implemented this in the right way.
By the way Visual Studio 2012 Release Candidate can be still downloaded from: http://msdn.microsoft.com/en-us/vstudio/bb984878.aspx. Maybe you should try it?