When you have a base class in your main executable and subclasses (say, plugins) defined in DLLs, what happens when you want to get a plugin?
I am looking for an article/answer that'd clarify what happens when you
load a DLL
call a DLL's function that returns a plugin* (it has virtual functions)
use that plugin in your main executable's code
delete, unload
I am thinking about the vtable and other C++ issues. For instance, if you unload the DLL stil having some plugins running... The "code" will be gone?
Since you are talking about plugins, you must be doing something like LoadLibrary. Assuming Windows:
Load a library. You would normally call LoadLibrary followed by GetProcAddress. The DLL is loaded into the process address space and you have the pointer to the function exposed.
Call a DLL's function that returns a plugin* (it has virtual functions). You would cast a the return value from GetProcAddress to the function pointer type, and call it. Everything should work as normal.
Use that plugin in your main executable's code. There is nothing special.
Delete the plugin. I assume you use a function in the DLL to do so, and it should be fine. Directly calling delete can be dangerous, as the DLL may have a separate memory manager (depending on what runtime you use).
Unload the DLL. You call FreeLibrary, and the code is gone. The pointers you previously get from GetProcAddress become invalid.
I am not aware specific vtable issues. If you unload the DLL while the code is still running, I would assume the program will crash at this point, as the code address space becomes invalid.
What's the best way for loading a dll from a dll ?
My problem is I can't load a dll on process_attach, and I cannot load the dll from the main program, because I don't control the main program source. And therefore I cannot call a non-dllmain function, too.
After all the debate that went on in the comments, I think that it's better to summarize my positions in a "real" answer.
First of all, it's still not clear why you need to load a dll in DllMain with LoadLibrary. This is definitely a bad idea, since your DllMain is running inside another call to LoadLibrary, which holds the loader lock, as explained by the documentation of DllMain:
During initial process startup or after a call to LoadLibrary, the system scans the list of loaded DLLs for the process. For each DLL that has not already been called with the DLL_PROCESS_ATTACH value, the system calls the DLL's entry-point function. This call is made in the context of the thread that caused the process address space to change, such as the primary thread of the process or the thread that called LoadLibrary. Access to the entry point is serialized by the system on a process-wide basis. Threads in DllMain hold the loader lock so no additional DLLs can be dynamically loaded or initialized.
The entry-point function should perform only simple initialization or termination tasks. It must not call the LoadLibrary or LoadLibraryEx function (or a function that calls these functions), because this may create dependency loops in the DLL load order. This can result in a DLL being used before the system has executed its initialization code. Similarly, the entry-point function must not call the FreeLibrary function (or a function that calls FreeLibrary) during process termination, because this can result in a DLL being used after the system has executed its termination code.
(emphasis added)
So, this on why it is forbidden; for a clear, more in-depth explanation, see this and this, for some other examples about what can happen if you don't stick to these rules in DllMain see also some posts in Raymond Chen's blog.
Now, on Rakis answer.
As I already repeated several times, what you think that is DllMain, isn't the real DllMain of the dll; instead, it's just a function that is called by the real entrypoint of the dll. This one, in turn, is automatically took by the CRT to perform its additional initialization/cleanup tasks, among which there is the construction of global objects and of the static fields of the classes (actually all these from the compiler's perspective are almost the same thing). After (or before, for the cleanup) it completes such tasks, it calls your DllMain.
It goes somehow like this (obviously I didn't write all the error checking logic, it's just to show how it works):
/* This is actually the function that the linker marks as entrypoint for the dll */
BOOL WINAPI CRTDllMain(
__in HINSTANCE hinstDLL,
__in DWORD fdwReason,
__in LPVOID lpvReserved
)
{
BOOL ret=FALSE;
switch(fdwReason)
{
case DLL_PROCESS_ATTACH:
/* Init the global CRT structures */
init_CRT();
/* Construct global objects and static fields */
construct_globals();
/* Call user-supplied DllMain and get from it the return code */
ret = DllMain(hinstDLL, fdwReason, lpvReserved);
break;
case DLL_PROCESS_DETACH:
/* Call user-supplied DllMain and get from it the return code */
ret = DllMain(hinstDLL, fdwReason, lpvReserved);
/* Destruct global objects and static fields */
destruct_globals();
/* Destruct the global CRT structures */
cleanup_CRT();
break;
case DLL_THREAD_ATTACH:
/* Init the CRT thread-local structures */
init_TLS_CRT();
/* The same as before, but for thread-local objects */
construct_TLS_globals();
/* Call user-supplied DllMain and get from it the return code */
ret = DllMain(hinstDLL, fdwReason, lpvReserved);
break;
case DLL_THREAD_DETACH:
/* Call user-supplied DllMain and get from it the return code */
ret = DllMain(hinstDLL, fdwReason, lpvReserved);
/* Destruct thread-local objects and static fields */
destruct_TLS_globals();
/* Destruct the thread-local CRT structures */
cleanup_TLS_CRT();
break;
default:
/* ?!? */
/* Call user-supplied DllMain and get from it the return code */
ret = DllMain(hinstDLL, fdwReason, lpvReserved);
}
return ret;
}
There isn't anything special about this: it also happens with normal executables, with your main being called by the real entrypoint, which is reserved by the CRT for the exact same purposes.
Now, from this it will be clear why the Rakis' solution isn't going to work: the constructors for global objects are called by the real DllMain (i.e. the actual entrypoint of the dll, which is the one about the MSDN page on DllMain talks about), so calling LoadLibrary from there has exactly the same effect as calling it from your fake-DllMain.
Thus, following that advice you'll obtain the same negative effects of calling directly LoadLibrary in the DllMain, and you'll also hide the problem in a seemingly-unrelated position, which will make the next maintainer work hard to find where this bug is located.
As for delayload: it may be an idea, but you must be really careful not to call any function of the referenced dll in your DllMain: in fact, if you did that you would trigger a hidden call to LoadLibrary, which would have the same negative effects of calling it directly.
Anyhow, in my opinion, if you need to refer to some functions in a dll the best option is to link statically against its import library, so the loader will automatically load it without giving you any problem, and it will resolve automatically any strange dependency chain that may arise.
Even in this case you mustn't call any function of this dll in DllMain, since it's not guaranteed that it's already been loaded; actually, in DllMain you can rely only on kernel32 being loaded, and maybe on dlls you're absolutely sure that your caller has already loaded before the LoadLibrary that is loading your dll was issued (but still you shouldn't rely on this, because your dll may also be loaded by applications that don't match these assumptions, and just want to, e.g., load a resource of your dll without calling your code).
As pointed out by the article I linked before,
The thing is, as far as your binary is concerned, DllMain gets called at a truly unique moment. By that time OS loader has found, mapped and bound the file from disk, but - depending on the circumstances - in some sense your binary may not have been "fully born". Things can be tricky.
In a nutshell, when DllMain is called, OS loader is in a rather fragile state. First off, it has applied a lock on its structures to prevent internal corruption while inside that call, and secondly, some of your dependencies may not be in a fully loaded state. Before a binary gets loaded, OS Loader looks at its static dependencies. If those require additional dependencies, it looks at them as well. As a result of this analysis, it comes up with a sequence in which DllMains of those binaries need to be called. It's pretty smart about things and in most cases you can even get away with not following most of the rules described in MSDN - but not always.
The thing is, the loading order is unknown to you, but more importantly, it's built based on the static import information. If some dynamic loading occurs in your DllMain during DLL_PROCESS_ATTACH and you're making an outbound call, all bets are off. There is no guarantee that DllMain of that binary will be called and therefore if you then attempt to GetProcAddress into a function inside that binary, results are completely unpredictable as global variables may not have been initialized. Most likely you will get an AV.
(again, emphasis added)
By the way, on the Linux vs Windows question: I'm not a Linux system programming expert, but I don't think that things are so different there in this respect.
There are still some equivalents of DllMain (the _init and _fini functions), which are - what a coincidence! - automatically took by the CRT, which in turn, from _init, calls all the constructors for the global objects and the functions marked with __attribute__ constructor (which are somehow the equivalent of the "fake" DllMain provided to the programmer in Win32). A similar process goes on with destructors in _fini.
Since _init too is called while the dll loading is still taking place (dlopen didn't return yet), I think that you're subject to similar limitations in what you can do in there. Still, in my opinion on Linux the problem is felt less, because (1) you have to explicitly opt-in for a DllMain-like function, so you aren't immediately tempted to abuse of it and (2), Linux applications, as far as I saw, tend to use less dynamic loading of dlls.
In a nutshell
No "correct" method will allow you to reference to any dll other than kernel32.dll in DllMain.
Thus, don't do anything important from DllMain, neither directly (i.e. in "your" DllMain called by the CRT) neither indirectly (in global class/static fields constructors), especially don't load other dlls, again, neither directly (via LoadLibrary) neither indirectly (with calls to functions in delay-loaded dlls, which trigger a LoadLibrary call).
The right way to have another dll loaded as a dependency is to - doh! - mark it as a static dependency. Just link against its static import library and reference at least one of its functions: the linker will add it to the dependency table of the executable image, and the loader will load it automatically (initializing it before or after the call to your DllMain, you don't need to know about it because you mustn't call it from DllMain).
If this isn't viable for some reason, there's still the delayload options (with the limits I said before).
If you still, for some unknown reason, have the inexplicable need to call LoadLibrary in DllMain, well, go ahead, shoot in your foot, it's in your faculties. But don't tell me I didn't warn you.
I was forgetting: another fundamental source of information on the topic is the [Best Practices for Creating DLLs][6] document from Microsoft, which actually talks almost only about the loader, DllMain, the loader lock and their interactions; have a look at it for additional information on the topic.
Addendum
No, not really an answer to my question. All it says is: "It's not possible with dynamic linking, you must link statically", and "you musn't call from dllmain".
Which *is* an answer to your question: under the conditions you imposed, you can't do what you want. In a nutshell of a nutshell, from DllMain you can't call *anything other than kernel32 functions*. Period.
Although in detail, but I'm not really interested in why it doesn't work,
You should, instead, because understanding why the rules are made in that way makes you avoid big mistakes.
fact is, the loader is not resolving dependenies correctly and the loading process is improperly threaded from Microsoft's part.
No, my dear, the loader does its job correctly, because *after* LoadLibrary has returned, all the dependencies are loaded and everything is ready to be used. The loader tries to call the DllMain in dependency order (to avoid problems with broken dlls which rely on other dlls in DllMain), but there are cases in which this is simply impossible.
For example, there may be two dlls (say, A.dll and B.dll) that depend on each other: now, whose DllMain is to call first? If the loader initialized A.dll first, and this, in its DllMain, called a function in B.dll, anything could happen, since B.dll isn't initialized yet (its DllMain hasn't been called yet). The same applies if we reverse the situation.
There may be other cases in which similar problems may arise, so the simple rule is: don't call any external functions in DllMain, DllMain is just for initializing the internal state of your dll.
The problem is there is no other way then doing it on dll_attach, and all the nice talk about not doing anything there is superfluous, because there is no alternative, at least not in my case.
This discussion is going on like this: you say "I want to solve an equation like x^2+1=0 in the real domain". Everybody says you that it's not possible; you say that it's not an answer, and blame the math.
Someone tells you: hey, you can, here's a trick, the solution is just +/-sqrt(-1); everybody downvotes this answer (because it's wrong for your question, we're going outside the real domain), and you blame who downvotes. I explain you why that solution is not correct according to your question and why this problem can't be solved in the real domain. You say that you don't care about why it can't be done, that you can only do that in the real domain and again blame math.
Now, since, as explained and restated a million times, under your conditions your answer has no solution, can you explain us why on earth do you "have" to do such an idiotic thing as loading a dll in DllMain? Often "impossible" problems arise because we've chosen a strange route to solve another problem, which brings us to deadlock. If you explained the bigger picture, we could suggest a better solution to it which does not involve loading dlls in DllMain.
PS: If I statically link DLL2 (ole32.dll, Vista x64) against DLL1 (mydll), which version of the dll will the linker require on older operating systems?
The one that is present (obviously I'm assuming you're compiling for 32 bit); if an exported function needed by your application isn't present in the found dll, your dll is simply not loaded (LoadLibrary fails).
Addendum (2)
Positive on injection, with CreateRemoteThread if you wanna know. Only on Linux and Mac the dll/shared library is loaded by the loader.
Adding the dll as a static dependency (what has been suggested since the beginning) makes it to be loaded by the loader exactly as Linux/Mac do, but the problem is still there, since, as I explained, in DllMain you still cannot rely on anything other than kernel32.dll (even if the loader in general intelligent enough to init first the dependencies).
Still, the problem can be solved. Create the thread (that actually calls LoadLibrary to load your dll) with CreateRemoteThread; in DllMain use some IPC method (for example named shared memory, whose handle will be saved somewhere to be closed in the init function) to pass to the injector program the address of the "real" init function that your dll will provide. DllMain then will exit without doing anything else. The injector application, instead, will wait for the end of the remote thread with WaitForSingleObject using the handle provided by CreateRemoteThread. Then, after the remote thread will be ended (thus LoadLibrary will be completed, and all the dependencies will be initialized), the injector will read from the named shared memory created by DllMain the address of the init function in the remote process, and start it with CreateRemoteThread.
Problem: on Windows 2000 using named objects from DllMain is prohibited because
In Windows 2000, named objects are provided by the Terminal Services DLL. If this DLL is not initialized, calls to the DLL can cause the process to crash.
So, this address may have to be passed in another manner. A quite clean solution would be to create a shared data segment in the dll, load it both in the injector application and in the target one and have it put in such data segment the required address. The dll would obviously have to be loaded first in the injector and then in the target, because otherwise the "correct" address would be overwritten.
Another really interesting method that can be done is to write in the other process memory a little function (directly in assembly) that calls LoadLibrary and returns the address of our init function; since we wrote it there, we can also call it with CreateRemoteThread because we know where it is.
In my opinion, this is the best approach, and is also the simplest, since the code is already there, written in this nice article. Have a look at it, it is quite interesting and it probably will do the trick for your problem.
The most robust way is to link the first DLL against the import lib of the second. This way, the actual loading of the second DLL will be done by Windows itself. Sounds very trivial, but not everyone knows that DLLs can link against other DLLs. Windows can even deal with cyclic dependencies. If A.DLL loads B.DLL which needs A.DLL, the imports in B.DLL are resolved without loading A.DLL again.
I suggest you to use delay-loading mechanism. The DLL will be loaded at the fisrt time you call imported function. Moreover you can modify load function and error handling. See Linker Support for Delay-Loaded DLLs for more info.
One possible answer is through the use of LoadLibrary and GetProcAddress to access pointers to functions found/located inside the loaded dll - but your intentions/needs aren't clear enough to determine if this is a suitable answer.
How can I ensure a dll is not unloaded while any objects in it exist?
The problem is, when I was using explict memory management I could delete the dll objects before freeing the dll, however with smart pointers I have no controll over the order there destroyed, meaning the dll may be freed first causeing a crash when trying to free one of the other objects:
FlPtr is a simple refrence counting class thats calls AddRef and Release as needed
ExampleDll *dll = LoadDll(L"bin\\example.dll");
IObject *obj = dll->CreateObject();
...
obj->Release();
delete dll;//fine because all objects already deleted
return 0;
auto_ptr<ExampleDll> dll = LoadDll(L"bin\\example.dll");
FlPtr<IObject> obj = dll->CreateObject();
...
return 0;//crash if dll is destructed before obj since Object::Release needs to call into the dll
I tried making the dll handle unloading itsself, ie only unload after all objects have been deleted. This work by creating a new object IExampleDll which the dll implements. This is like the ExampleDll object from before but lives in the dll rather than the exe and is also refrence counted. Each object in the dll increments this refrence on contruction and deincrements it on destruction. This means the refrence count only reaches zero when the exe has Released its refrences AND all the dlls objects have been destroyed. It then deletes itsself calling FreeLibrary(GetModuleHandle()) in its destructor.
This however crashes at the FreeLibrary, im asuming because the thread is still in the dlls code that is being unloaded...
I'm at a loss now how to make sure the dll is only unloaded when there are no remaining objects, apart from going back to freeing the dll explicitly after everything else should have been deleted;
int main()
{
ExampleDll *dll = LoadDll("bin\\example.dll");
restOfProgram();
delete dll;
}
This approach becomes difficult when dlls need to be loaded/unloaded mid program saftly, ie if the user changed from d3d to openGL in options.
Assuming you do not want to terminate the thread when unloading the library (otherwise, see MSalters), you need to free the library from the caller that loaded it.
COM solves that by an in-DLL instance counter (much like yours, if I understand you correctly), and regulary checking it by calling a global exported CanUnloadNow function.
Another option is to have your object/interface smart pointers ALSO reference the DLL they came from. This would increase the client data size, but you wouldn't need to touch the DLL. You might even recycle the LoadLibrary/FreeLibrary reference counter, however that might hit performance.
Also, none of these schemes help much if you get circular DLL dependencies (Component DllA.X references DllB.Y, which references DllA.Z). I haven't yet fould a good solution to that that doesn#t requrie global knowledge.
For the case where the DLL is switched at run time, I'd avoid the smart pointer system for objects created by the DLL and use a system like this:
|-----------------------| |--------------------------|
| Abstraction Interface | | Implementation Interface |
|-----------------------| |--------------------------|
^ ^
| |
|-------------|1 *|-------------------|* *|----------------|
| Application |-------| Abstraction Layer |--------| Implementation |
|-------------| |-------------------| |----------------|
\------------- Main Program ------------------/ \-------- DLL --------/
The application holds a list of all the allocated abstraction layer objects. The abstraction layer objects are the only objects that are allowed to own pointers to objects created by the implementation layer. When swapping DLLs, first iterate all abstraction layer objects and tell them to release the implementation specific data. Then unload the DLL and load the new DLL. Then iterate the abstraction layer objects again and tell them to create new implementation specific data.
MSDN is explicit on this topic: "A thread that must unload the DLL in which it is executing and then terminate itself should call FreeLibraryAndExitThread instead of calling FreeLibrary and ExitThread separately. Otherwise, a race condition can occur. For details, see the Remarks section of FreeLibraryAndExitThread.
Ok I think the best choice is to use the COM approach of polling the dll to see if it can be unloaded. How can I go about doing this though so that I can continue polling the dll after everything else has closed down (ie the main thread has terminated)? Do I need to create a completly seperate process to do this, in which case how do I do it so this seperate process knows about all the loaded dlls, and in a way which has VERY little impact on proformance?
Mayby I could just create the polling system when all the DllPtr's are out of scope and terminate it as soon as it has free the dll? That way it only exists for as long as it takes for any remaing smart pointers to be destroyed.
Let's say I write a DLL in C++, and declare a global object of a class with a non-trivial destructor. Will the destructor be called when the DLL is unloaded?
In a Windows C++ DLL, all global objects (including static members of classes) will be constructed just before the calling of the DllMain with DLL_PROCESS_ATTACH, and they will be destroyed just after the call of the DllMain with DLL_PROCESS_DETACH.
Now, you must consider three problems:
0 - Of course, global non-const objects are evil (but you already know that, so I'll avoid mentionning multithreading, locks, god-objects, etc.)
1 - The order of construction of objects or different compilation units (i.e. CPP files) is not guaranteed, so you can't hope the object A will be constructed before B if the two objects are instanciated in two different CPPs. This is important if B depends on A. The solution is to move all global objects in the same CPP file, as inside the same compilation unit, the order of instanciation of the objects will be the order of construction (and the inverse of the order of destruction)
2 - There are things that are forbidden to do in the DllMain. Those things are probably forbidden, too, in the constructors. So avoid locking something. See Raymond Chen's excellent blog on the subject:
Some reasons not to do anything scary in your DllMain
Another reason not to do anything scary in your DllMain: Inadvertent deadlock
Some reasons not to do anything scary in your DllMain, part 3
In this case, lazy initialization could be interesting: The classes remain in an "un-initialized" state (internal pointers are NULL, booleans are false, whatever) until you call one of their methods, at which point they'll initialize themselves. If you use those objects inside the main (or one of the main's descendant functions), you'll be ok because they will be called after execution of DllMain.
3 - Of course, if some global objects in DLL A depend on global objects in DLL B, you should be very very careful about DLL loading order, and thus dependancies. In this case, DLLs with direct or indirect circular dependancies will cause you an insane amount of headaches. The best solution is to break the circular dependancies.
P.S.: Note that in C++, constructor can throw, and you don't want an exception in the middle of a DLL loading, so be sure your global objects won't be using exception without a very, very good reason. As correctly written destructors are not authorized to throw, the DLL unloading should be ok in this case.
This page from Microsoft goes into the details of DLL initialization and destruction of globals:
http://msdn.microsoft.com/en-us/library/988ye33t.aspx
If you want to see the actual code that gets executed when linking a .dll, take a look at %ProgramFiles%\Visual Studio 8\vc\crt\src\dllcrt0.c.
From inspection, destructors will be called via _cexit() when the internal reference count maintained by the dll CRT hits zero.
It should be called when either the application ends or the DLL is unloaded, whichever comes first. Note that this is somewhat dependent on the actual runtime you're compiling against.
Also, beware non-trivial destructors as there are both timing and ordering issues. Your DLL may be unloaded after a DLL your destructor relies on, which would obviously cause issues.
In windows binary image files with extension *.exe, *.dll are in PE format
Such files have Entry Point. You can view it with dumpbin tool like
dumpbin /headers dllname.dll
If you use C runtime from Microsoft, then your entry point will be something like
*CRTStartup or *DllMainCRTStartup
Such functions perform initialization of c and c++ runtime and delegate execution to (main, WinMain) or to DllMain respectively.
If you use Microsofts VC compiler then you can watch at source code of this functions in yours VC directory:
crt0.c
dllcrt0.c
DllMainCRTStartup process all things need to init/deinit your global variables from .data sections in normal scenario, when it retrive notification DLL_PROCESS_DETACH during dll unload. For example:
main or WinMain of startup thread of program returns control flow
you explictly call FreeLibrary and use-dll-counter is zero
When DllMain with fdwReason = DLL_PROCESS_DETACH parameter is called it means the DLL is unloaded by the application. This is the time before the destructor of global/static objects gets called.