This program's stdafx.h is a little like this below.
// ...
#import "./lib/64/msado15.dll" rename("EOF", "EndOfFile") no_namespace
// ...
The program works okay and there are no problems.
I was curious about what would happen if I deleted msado15.dll.
So, I deleted it, and the program still works fine.
I assumed that why the program works without msado15.dll in the same directory must be the dll file is loaded somewhere else.
To make sure exactly where the dll is loaded, I used "Dependancy Walker", and I found out this program is not loading msado15.dll at all.
I would be glad if I could figure out what I'm missing.
Thanks in advance.
Dependency Walker can only see static DLL dependencies (where there's a reference in the EXE's imports table). If a DLL is loaded at runtime via LoadLibrary (or by delay loading -- using /delayload), Dependency Walker will not be able to see this.
#import doesn't actually impose a static dependency on the DLL. It loads the COM typelibrary information from the DLL at compile time. This is transformed into C++ bindings for the COM classes and interfaces defined in the typelibrary. You can see these as .tli and .tlh files in your Debug or Release directory. You might be able to remove the DLL file, and -- as long as these files still exist -- VS might continue to build your project successfully.
Similarly, at runtime, since #import doesn't actually impose a static dependency on the DLL (that is: it won't add it to the imports table in the EXE), Dependency Walker will be unable to see this dependency.
At runtime, however, the EXE will call (indirectly) LoadLibrary, and (if the DLL is missing) this will cause a failure at runtime, which your program might deal with appropriately, or which might cause your program to crash.
Even with all that said, #import merely imports a COM typelibrary. The COM typelibrary defines the interfaces used in the COM objects, along with the CLSID values of those objects. In order to find the code that implements the COM objects, the CLSID values are resolved using the registry (in HKEY_CLASSES_ROOT\CLSID. The code implementing the COM objects may not actually be in the same binary as the original typelibrary.
This means that the DLL may not even be required at runtime. I think that this is rare, however.
Moreover, the COM objects may be implemented as out-of-process objects (meaning that another EXE is loaded). In this case, all that's loaded into your process will be the proxy/stub DLL configured for the relevant interfaces. Often, these will be defined using the TLB (in which case, your #import-ed DLL will be loaded); but they may be implemented in a completely different DLL. Either way, the DLL will be loaded dynamically, which means that Dependency Walker won't see it.
To see which DLLs are loaded by the process, you will need something like SysInternals Process Monitor or Process Explorer.
Related
Can you help me to understand, why do we need .lib files when importing functions and data from dll?
I've heard, that it contains a list of the exported functions and data elements from the corresponding dll, but when I used CFF Explorer to explore my dll, I found out that dll already has addresses of exporting functions so I theoretically can link my program with .dll without any additional files.
Can you, please, explain what kind of data is stored in the .lib files more detailed.
And, also, yes, I know, that visual studio forces us to add .lib files into additional dependencies section, but why does it really needs them?
When your source code statically calls exported DLL functions, or statically accesses exported DLL variables, those references are compiled into your executable's intermediate object files as pointers, whose values get populated at run-time.
When the linker is combining the compiler-generated object files to make the final executable, it has to figure out what all of the compiler-generated references actually refer to. If it can't match a given reference to some piece of code in your executable, it needs to match it to an external DLL instead. So it needs to know which DLLs to even look at, and how those DLLs export things. A DLL may export a given function/variable by name OR by ordinal number, so the linker needs a way to map the identifiers used by your code references to specific entries in the EXPORTS tables of specific .dll files (especially in the case where things are exported by ordinals). Static-link .lib files provide the linker with that mapping information (ie FunctionA maps to Ordinal 123 in DLL XYZ.dll, FunctionB maps to name _FunctionB#4 in DLL ABC.dll, etc).
The linker can then populate the IMPORTS table of your executable with information about the appropriate EXPORTS entries needed, and then make the DLL references in your code point to the correct IMPORTS entries (if the linker can't resolve a compiler-generate reference to a piece of code in your executable, or to a specific DLL export, it aborts with an "unresolved external" error).
Then, when your executable is loaded at run-time, the OS Loader looks at the IMPORTS table to know which DLL exports are needed, so it can then load the appropriate DLLs into memory and update the entries in the IMPORTS table with real memory addresses that are based on each DLL's EXPORTS table (if a referenced DLL fails to load, or if a referenced export fails to be found, the OS Loader aborts loading your executable). That way, when your code calls DLL functions or accesses DLL variables, those accesses go to the right places.
Things are very different if your source code dynamically accesses DLL functions/variables via explicit calls to GetProcAddress() at run-time. In that case, static-link .lib files are not needed for those accesses, since your own code is handling the loading of DLLs into memory and locating the exports that it wants to use.
However, there is a 3rd option that blends the above scenarios together: you can write your code to access the DLL functions/variables statically but use your linker's delay-load feature (if it has one). In that case, you still need static-link .lib files for each delay-loaded DLL you access, but the linker populates a separate DELAYLOAD table in your executable with references to the DLL exports, instead of populating the IMPORTS table. It points the compiler-generated DLL references to stubs in your compiler's RTL that will replace the references with addresses from GetProcAddress() when the stubs are accessed for the first time at run-time, thus avoiding the need for the references to be populated by the OS Loader at load-time. This allows your executable to run normally even if the DLL exports are not present at load-time, and may not even need to load the DLLs at all if they are never used (of course, if your executable does try to access a DLL export dynamically and it fails to load, your code is likely to crash, but that is a separate issue).
I've heard, that it contains a list of the exported functions and data elements from the corresponding dll, but when I used CFF Explorer to explore my dll, I found out that dll already has addresses of exporting functions so I theoretically can link my program with .dll without any additional files.
As a trivial example of why this can't always work, consider an executable that accesses two DLLs, one for a Winsock filter and the other for an allocator. And say that on this particular machine, the Winsock filter DLL happens to also implement an allocator with the same API and the allocator DLL happens to also implement a Winsock filter with the same API. How could the compiler know which API functions to access from which DLL? The library file contains the intent in accessing the DLL, that is, the API and functions you want to access.
Importantly, there is no such thing as "The corresponding DLL". There might be different DLL files on different systems. What the linker needs to know is what the DLL is supposed to look like that it can rely on, not what the DLL that you might happen to use on some particular system might happen to be.
For example, suppose the DLL file contains an allocator. You might have one DLL file for an allocator with debugging, one for an allocator with optimizations for specific CPU versions, and one for an allocator that uses a new, experimental algorithm. What the linker needs to know is the API that all these DLL files implement, not the specific implementation in any one file.
You can produce a LIB file from a DLL file but you might wind up building an executable that doesn't work when using some other version of the DLL file. You would have to assume that whatever this particular DLL happens to do is precisely what every other DLL that implements the same API will happen to do.
I'm using MYSQL library, and libmysql.lib /.dll.
My program cannot be working without the libmysql.dll
When I'm trying to run my project without the dll I'm getting that error message.
What I'm basically want to do is to put that dll in my .exe file.
build the .exe file with that dll and make the program read it from himself.
I mean, give the program to people with that dll inside.
It is possible ?
I tried this section: embed DLL in MFC C++ EXE?
But the program still asking for the dll .. (But I do see that the size of the .exe has been changed) so that dll has been added.
But the program still asking for the libmysql.dll ..
All the point is to use it inside the .exe file..
thanks.
What you are asking for cannot be done if you statically link to the DLL at compile-time. You need to dynamically link to the DLL at run-time instead, either by explicit calls to LoadLibrary() and GetProcAddress() to access the DLL functions directly, or by utilizing your compiler's delay-load functionality (which uses LoadLibrary() and GetProcAddress() internally, but hides that fact from your code). Either way, you can then store the DLL in your EXE's resources at compile-time, then extract the resource to a temporary file at run-time and load/use it as needed (you can't use the DLL from inside the EXE's resources directly. Well, there is a way to do it, but it is a VERY complicated and advanced technique, as it requires implementing your own executable loader that basically mimics what the OS's built-in executable loader already does). When you are done using the DLL, you can unload it from memory and delete the temp file.
in my Windows C++ program I have a few dependencies on DLLs (coming with drivers of input devices). I don't actually load the DLLs myself, but the drivers provide (small) .lib library that I statically link against (and I assume it is those libraries that make sure the DLLs are present in the system and loads them). I'm writing an application that can take input from a series of video cameras. At run-time, the user chooses which one to use. Currently my problem is that my routines that query whether a camera is connected already require the functionality of the camera being present on the system. I.e. let's say there is camera model A and B, the user has to install the drivers for A and B, even if he knows he just owns model B. The user has to do this, as otherwise my program won't even start (then when it started it will of course tell the user which of the two cameras are actually connected).
I'd like to know whether there is any possibility, at run-time, to determine which of the DLLs are present, and for those that aren't, somehow disable loading even the static (and, thus, dynamic) component.
So basically my problem is that you cannot do if(DLL was found){ #include "source that includes header using functions defined in lib which loads DLL"}
I think using the DELAYLOAD linker flag may provide the functionality required. It would allow linking with the .lib files but would only attempt to load the DLL if it is used:
link.exe ... /DELAYLOAD:cameraA.dll /DELAYLOAD:cameraB.dll Delayimp.lib
The code would be structured something similar to:
if (/* user selected A */)
{
// Use camera A functions, resulting in load of cameraA's DLL.
}
else
{
// Use camera B functions, resulting in load of cameraB's DLL.
}
From Linker Support for Delay-Loaded DLLs
:
Beginning with Visual C++ 6.0, when statically linking with a DLL, the
linker provides options to delay load the DLL until the program calls
a function in that DLL.
An application can delay load a DLL using the /DELAYLOAD (Delay Load Import)
linker option with a helper function (default implementation provided by
Visual C++). The helper function will load the DLL at run time by calling
LoadLibrary and GetProcAddress for you.
You should consider delay loading a DLL if:
- Your program may not call a function in the DLL.
- A function in the DLL may not get called until late in your program's
execution.
You need to load the libs at run-time. Take a look a look at LoadLibary.
This is an MSDN article about this: DLLs the Dynamic Way I just glanced over this. It's very old.
This one shows the usage of LoadLibrary: Using Run-Time Dynamic Linking
I know this may seem quite basic to geeks. But I want to make it crystal clear.
When I want to use a Win32 DLL, usually I just call the APIs like LoadLibrary() and GetProcAdderss(). But recently, I am developing with DirectX9, and I need to add d3d9.lib, d3dx9.lib, etc files.
I have heard enough that LIB is for static linking and DLL is for dynamic linking.
So my current understanding is that LIB contains the implementation of the methods and is statically linked at link time as part of the final EXE file. While DLL is dynamic loaded at runtime and is not part of the final EXE file.
But sometimes, there're some LIB files coming with the DLL files, so:
What are these LIB files for?
How do they achieve what they are meant for?
Is there any tools that can let me inspect the internals of these LIB files?
Update 1
After checking wikipedia, I remember that these LIB files are called import library.
But I am wondering how it works with my main application and the DLLs to be dynamically loaded.
Update 2
Just as RBerteig said, there're some stub code in the LIB files born with the DLLs. So the calling sequence should be like this:
My main application --> stub in the LIB --> real target DLL
So what information should be contained in these LIBs? I could think of the following:
The LIB file should contain the fullpath of the corresponding DLL; So the DLL could be loaded by the runtime.
The relative address (or file offset?) of each DLL export method's entry point should be encoded in the stub; So correct jumps/method calls could be made.
Am I right on this? Is there something more?
BTW: Is there any tool that can inspect an import library? If I can see it, there'll be no more doubts.
Linking to a DLL file can occur implicitly at compile link time, or explicitly at run time. Either way, the DLL ends up loaded into the processes memory space, and all of its exported entry points are available to the application.
If used explicitly at run time, you use LoadLibrary() and GetProcAddress() to manually load the DLL and get pointers to the functions you need to call.
If linked implicitly when the program is built, then stubs for each DLL export used by the program get linked in to the program from an import library, and those stubs get updated as the EXE and the DLL are loaded when the process launches. (Yes, I've simplified more than a little here...)
Those stubs need to come from somewhere, and in the Microsoft tool chain they come from a special form of .LIB file called an import library. The required .LIB is usually built at the same time as the DLL, and contains a stub for each function exported from the DLL.
Confusingly, a static version of the same library would also be shipped as a .LIB file. There is no trivial way to tell them apart, except that LIBs that are import libraries for DLLs will usually be smaller (often much smaller) than the matching static LIB would be.
If you use the GCC toolchain, incidentally, you don't actually need import libraries to match your DLLs. The version of the Gnu linker ported to Windows understands DLLs directly, and can synthesize most any required stubs on the fly.
Update
If you just can't resist knowing where all the nuts and bolts really are and what is really going on, there is always something at MSDN to help. Matt Pietrek's article An In-Depth Look into the Win32 Portable Executable File Format is a very complete overview of the format of the EXE file and how it gets loaded and run. Its even been updated to cover .NET and more since it originally appeared in MSDN Magazine ca. 2002.
Also, it can be helpful to know how to learn exactly what DLLs are used by a program. The tool for that is Dependency Walker, aka depends.exe. A version of it is included with Visual Studio, but the latest version is available from its author at http://www.dependencywalker.com/. It can identify all of the DLLs that were specified at link time (both early load and delay load) and it can also run the program and watch for any additional DLLs it loads at run time.
Update 2
I've reworded some of the earlier text to clarify it on re-reading, and to use the terms of art implicit and explicit linking for consistency with MSDN.
So, we have three ways that library functions might be made available to be used by a program. The obvious follow up question is then: "How to I choose which way?"
Static linking is how the bulk of the program itself is linked. All of your object files are listed, and get collected together in to the EXE file by the linker. Along the way, the linker takes care of minor chores like fixing up references to global symbols so that your modules can call each other's functions. Libraries can also be statically linked. The object files that make up the library are collected together by a librarian in a .LIB file which the linker searches for modules containing symbols that are needed. One effect of static linking is that only those modules from the library that are used by the program are linked to it; other modules are ignored. For instance, the traditional C math library includes many trigonometry functions. But if you link against it and use cos(), you don't end up with a copy of the code for sin() or tan() unless you also called those functions. For large libraries with a rich set of features, this selective inclusion of modules is important. On many platforms such as embedded systems, the total size of code available for use in the library can be large compared to the space available to store an executable in the device. Without selective inclusion, it would be harder to manage the details of building programs for those platforms.
However, having a copy of the same library in every program running creates a burden on a system that normally runs lots of processes. With the right kind of virtual memory system, pages of memory that have identical content need only exist once in the system, but can be used by many processes. This creates a benefit for increasing the chances that the pages containing code are likely to be identical to some page in as many other running processes as possible. But, if programs statically link to the runtime library, then each has a different mix of functions each laid out in that processes memory map at different locations, and there aren't many sharable code pages unless it is a program that all by itself is run in more than process. So the idea of a DLL gained another, major, advantage.
A DLL for a library contains all of its functions, ready for use by any client program. If many programs load that DLL, they can all share its code pages. Everybody wins. (Well, until you update a DLL with new version, but that isn't part of this story. Google DLL Hell for that side of the tale.)
So the first big choice to make when planning a new project is between dynamic and static linkage. With static linkage, you have fewer files to install, and you are immune from third parties updating a DLL you use. However, your program is larger, and it isn't quite as good citizen of the Windows ecosystem. With dynamic linkage, you have more files to install, you might have issues with a third party updating a DLL you use, but you are generally being friendlier to other processes on the system.
A big advantage of a DLL is that it can be loaded and used without recompiling or even relinking the main program. This can allow a third party library provider (think Microsoft and the C runtime, for example) to fix a bug in their library and distribute it. Once an end user installs the updated DLL, they immediately get the benefit of that bug fix in all programs that use that DLL. (Unless it breaks things. See DLL Hell.)
The other advantage comes from the distinction between implicit and explicit loading. If you go to the extra effort of explicit loading, then the DLL might not even have existed when the program was written and published. This allows for extension mechanisms that can discover and load plugins, for instance.
These .LIB import library files are used in the following project property, Linker->Input->Additional Dependencies, when building a bunch of dll's that need additional information at link time which is supplied by the import library .LIB files. In the example below to not get linker errors I need to reference to dll's A,B,C, and D through their lib files. (note for the linker to find these files you may need to include their deployment path in Linker->General->Additional Library Directories else you will get a build error about being unable to find any of the provided lib files.)
If your solution is building all dynamic libraries you may have been able to avoid this explicit dependency specification by relying instead on the reference flags exposed under the Common Properties->Framework and References dialog. These flags appear to automatically do the linking on your behalf using the *.lib files.
This however is as it says a Common Properties, which is not configuration or platform specific. If you need to support a mixed build scenario as in our application we had a build configuration to render a static build and a special configuration that built a constrained build of a subset of assemblies that were deployed as dynamic libraries. I had used the Use Library Dependency Inputs and Link Library Dependencies flags set to true under various cases to get things to build and later realizing to simplify things but when introducing my code to the static builds I introduced a ton of linker warnings and the build was incredibly slow for the static builds. I wound up introducing a bunch of these sort of warnings...
warning LNK4006: "bool __cdecl XXX::YYY() already defined in CoreLibrary.lib(JSource.obj); second definition ignored D.lib(JSource.obj)
And I wound up using the manual specification of Additional Dependencies to satisfy the linker for the dynamic builds while keeping the static builders happy by not using a common property that slowed them down. When I deploy the dynamic subset build I only deploy the dll files as these lib files are only used at link time, not at runtime.
Here are some related MSDN topics to answer my question:
Linking an Executable to a DLL
Linking Implicitly
Determining Which Linking Method to Use
Building an Import Library and Export File
There are three kinds of libraries: static, shared and dynamically loaded libraries.
The static libraries are linked with the code at the linking phase, so they are actually in the executable, unlike the shared library, which has only stubs (symbols) to look for in the shared library file, which is loaded at run time before the main function gets called.
The dynamically loaded ones are much like the shared libraries, except they are loaded when and if the need arises by the code you've written.
In my mind, there are two method to link dll to exe.
Use dll and the import library (.lib file) implicitly
Use functions like loadlibrary() explicitly
Just curious, I was told that with dll files, you can make modifications to the dll without recompiling the whole application that uses it. On the other hand .lib files need to be compiled so the code can linked to the application as one.
So I know that the .lib files are turned into machine code. but what about the dll's ?? Are they turned into machine code upon the execution of the application ??
This could probably lead to easy hacking if not used right.
The dlls are still machine code. They're just dynamically linked in at run time (hence then name) so (if you don't change the function signatures) you don't have to recompile your main program to use a dll after it's been changed. A static library is physically part of your executable, that's why changes there require a recompile (or really, a relink).
DLLs do contain compiled machine code. The difference is that the linking between the application EXE and the DLL is done at runtime, instead of at (traditional) link time between OBJ and LIB files.
A DLL normally contains machine code. The point isn't that you can modify the DLL because it's source code, but that you can modify the source code for the DLL, re-compile and re-link, and as long as the interface remains the same, you can replace the old DLL with a new one, and the rest of the application can use the new instead of the old.
In reality, this frequently leads to problems, such as an application including code that depends on a bug in the old DLL. When/if you create a version that fixes the bug, it breaks the application. Google for "DLL Hell" for many more examples.
Simply put:DLLs can be swapped out post-compilation because they are physically separate from your exe. Lib and obj files, on the other hand, are
compiled into your exe, so that updating them
requires recompiling your app.
Dlls are effectively exes that don't define main().
When a DLL is linked to the main program, only the interface, i.e. classes, functions etc. it exports are linked by their signature. The actual machine code inside the DLL is loaded only on runtime, not on compile-time; that's why you can build your app with just the DLL's lib, and put the actual DLL where you'd rather have it (for example some shared DLLs folder or whatever).
You can then swap out this DLL with any other DLL with the same interface. That is how plugin-systems usually work, as each plugin is simply a DLL that conforms to a documented pre-defined interface, and the program just loads all DLLs of that interface it finds in some "plugins" directory.
The (possibly) confusing part here is that there are effectively two kind of .lib files:
a) libs for static linking. These put all their compiled code directly into the main app, and are a fixed part of the resulting .exe file.
b) libs for dynamic linking. These contain an interface for a DLL containing the actual code, which must be available to the app only at runtime (and if it isn't, it won't start, and simply tell you which DLL it couldn't find).
Btw: you don't have to specify which one of the libs you link to is what type, it does it automatically.
Also: Some applications are built as DLLs meant to being run inside some external environment. Webservices, for example, are implemented as DLLs with a specific interface, which are being run by an Apache/IISAPI/whatever server. This works similarly to the aforementioned plugin-system, but here each DLL effectively is the application.