I'm try to build a dynamic dll library on on windows using visual studio, but there are two file generated, one is dll file, another is .lib file;
My knowledege of using dll library is privode it to linker, I don't know what is ther purpose of .lib file, it has the same file extension as static lib, and it definitely is not static lib;
Does visual studio generate a .lib file when create dynamic dll library at all situtation?
Do I need to use the .lib file in my application?
How do I do use the .lib file in my application?
It has to do with the difference between "implicit" linking and "explicit" linking. The one sentence answer to your question is that that .lib file, often called a "stub lib" and officially called an "import library", is necessary when you do implicit linking but not otherwise.
In implicit linking, at compile time the compiler generates external function references for calls into the DLL, then the linker needs to link those to something: it uses the import library to help it do that. The linker treats these calls as special cases; it inserts code in the executable file for finding the DLL and for calling into the DLL for specific functions.
From the point-of-view of the programmer, implicit linking behaves like static linking except the DLL needs to be somewhere where the system can find it when the application is run or the system will pop up an error dialog at application startup.
Explicit linking means that it is the programmer's responsibility to find and load the DLL and to know what functions are in the DLL and how to call into them. See LoadLibrary and GetProcAddress for details. Explicit linking is useful if usage of a library is conditional or for applications with plugin architectures in which the actual DLLs to be used may not even be known at compile time.
In general though, if implicit linking does what you need, it is easier to just do it that way.
How does dynamic linking work generally?
On Windows (LoadLibrary), you need a .dll to call at runtime, but at link time, you need to provide a corresponding .lib file or the program won't link... What does the .lib file contain? A description of the .dll methods? Isn't that what the headers contain?
Relatedly, on *nix, you don't need a lib file... How how does the compiler know that the methods described in the header will be available at runtime?
As a newbie, when you think about either one of the two schemes, then the other, neither of them make sense...
To answer your questions one by one:
Dynamic linking defers part of the linking process to runtime.
It can be used in two ways: implicitly and explicitly.
Implicitly, the static linker will insert information into the
executable which will cause the library to load and resolve the
necessary symbols. Explicitly, you must call LoadLibrary or
dlopen manually, and then GetProcAddress/dlsym for each
symbol you need to use. Implicit loading is used for things
like the system library, where the implementation will depend on
the version of the system, but the interface is guaranteed.
Explicit loading is used for things like plug-ins, where the
library to be loaded will be determined at runtime.
The .lib file is only necessary for implicit loading. It
contains the information that the library actually provides this
symbol, so the linker won't complain that the symbol is
undefined, and it tells the linker in what library the symbols
are located, so it can insert the necessary information to cause
this library to automatically be loaded. All the header files
tell the compiler is that the symbols will exist, somewhere; the
linker needs the .lib to know where.
Under Unix, all of the information is extracted from the
.so. Why Windows requires two separate files, rather than
putting all of the information in one file, I don't know; it's
actually duplicating most of the information, since the
information needed in the .lib is also needed in the .dll.
(Perhaps licensing issues. You can distribute your program with
the .dll, but no one can link against the libraries unless
they have a .lib.)
The main thing to retain is that if you want implicit loading,
you have to provide the linker with the appropriate information,
either with a .lib or a .so file, so that it can insert that
information into the executable. And that if you want explicit
loading, you can't refer to any of the symbols in the library
directly; you have to call GetProcAddress/dlsym to get their
addresses yourself (and do some funny casting to use them).
The .lib file on Windows is not required for loading a dynamic library, it merely offers a convenient way of doing so.
In principle, you can use LoadLibrary for loading the dll and then use GetProcAddress for accessing functions provided by that dll. The compilation of the enclosing program does not need to access the dll in that case, it is only needed at runtime (ie. when LoadLibrary actually executes). MSDN has a code example.
The disadvantage here is that you need to manually write code for loading the functions from the dll. In case you compiled the dll yourself in the first place, this code simply duplicates knowledge that the compiler could have extracted from the dll source code automatically (like the names and signatures of exported functions).
This is what the .lib file does: It contains the GetProcAddress calls for the Dlls exported functions, generated by the compiler so you don't have to worry about it. In Windows terms, this is called Load-Time Dynamic Linking, since the Dll is loaded automatically by the code from the .lib file when your enclosing program is loaded (as opposed to the manual approach, referred to as run-time dynamic linking).
How does dynamic linking work generally?
The dynamic link library (aka shared object) file contains machine code instructions and data, along with a table of metadata saying which offsets in that code/data relate to which "symbols", the type of the symbol (e.g. function vs data), the number of bytes or words in the data, and a few other things. Different OS will tend to have different shared object file formats, and indeed the same OS may support several, but that's the gist of it.
So, imagine the shared library's a big chunk of bytes with an index like this:
SYMBOL ADDRESS TYPE SIZE
my_function 1000 function 2893
my_number 4800 variable 4
In general, the exact type of the symbols need not be captured in the metadata table - it's expected that declarations in the library's header files contain all the missing information. C++ is a bit special - compared to say C - because overloading can mean there are several functions with the same name, and namespaces allow for further symbols that would otherwise be ambiguously named - for that reason name mangling is typically used to concatenate some representation of the namespace and function arguments to the function name, forming something that can be unique in the library object file.
A program wanting to use the shared object can generally do one of two things:
have the OS load both itself and the shared object around the same time (before executing main()), with the OS Loader responsible for finding the symbols and examining metadata in the program file image about the use of those symbols, then patching in symbol addresses in the memory the program uses, such that the program can then just run and work functionally as if it'd known about the symbol addresses when it was first compiled (but perhaps a little slower)
or, explicitly in its own source code call dlopen sometime after main runs, then use dlsym or similar to get the symbol addresses, save them into (function/data) pointers based on the programmer's knowledge of the expected data types, then call them explicitly using the pointers.
On Windows (LoadLibrary), you need a .dll to call at runtime, but at link time, you need to provide a corresponding .lib file or the program won't link...
That doesn't sound right. Should be one or the other I'd think.
Wtf does the .lib file contain? A description of the .dll methods? Isn't that what the headers contain?
A lib file is - at this level of description - pretty much the same as a shared object file... the main difference is that the compiler's finding the symbol addresses before the program's shipped and run.
Modern *nix systems derive process of dynamic linking from Solaris OS. Linux, particularly, doesn't need separate .lib file because all external dependencies are contained in ELF format. .interp section of ELF file indicates that there are external symbols inside this executable that needed to be resolved dynamically. This comes for dynamic linking.
There is a way to handle dynamic linking in user space. This method is called dynamic loading. This is when you are using system calls to get function pointers to methods from external *.so.
More information can be found from this article http://www.ibm.com/developerworks/library/l-dynamic-libraries/.
Relatedly, on OS X (and I assume *nix... dlopen), you don't need a lib file... How how does the compiler know that the methods described in the header will be available at runtime?
Compilers or linkers do not need such information. You, the programmer, need to handle the situation that the shared libraries you try to open by dlopen() may not exist.
You can use a DLL file in Windows in two ways: Either you link with it, and you're done, nothing more to do. Or you load it dynamically during run-time.
If you link with it, then the DLL library file is used. The link-library contains information that the linker uses to actually know which DLL to load and where in the DLL functions are, so it can call them. When your program is loaded, the operating system also loads the DLL for you, basically what is does it call LoadLibrary for you.
In other operating systems (like OS X and Linux) it works in a similar way. The difference is that on these systems the linker can look directly at the dynamic library (the .so/.dynlib file) and figure out what's needed without a separate static library like on Windows.
To load a library dynamically, you don't need to link with anything related to the library you want to load.
Like others already said: what is included in a .lib file on Windows is included directly in the .so/.dynlib on Linux/OS X. But the main question is... why?
Isn't *nix solution better?
I think it is, but the .lib has one advantage. The developer linking to the DLL doesn't actually need to have access to the DLL file itself.
Does a scenario like that happen often in the real world? Is it worth the effort of maintaining two files per DLL file? I don't know.
Edit: Ok, guys let's make things even more confusing! You can link directly to a DLL on Windows, using MinGW. So the whole import library problem is not directly related to Windows itself. Taken from sampleDLL article from MinGW wiki:
The import library created by the "--out-implib" linker option is
required iff (==if and only if) the DLL shall be interfaced from some
C/C++ compiler other than the MinGW toolchain. The MinGW toolchain is
perfectly happy to directly link against the created DLL. More details
can be found in the ld.exe info files that are part of the binutils
package (which is a part of the toolchain).
Linux also requires to link, but instead against a .Lib library it needs to link to the dynamic linker /lib/ld-linux.so.2, but this usually happens behind the scenes when using GCC (however if using an assembler you do need to specify it manually).
Both approaches, either the Windows .LIB approach or the Linux dynamic linker linking approach, are considered in reality as static linking. There is, however, a difference that in Windows part of the work is done at link time although it still has work at load time (I am not sure, but I think that the .LIB file is merely for the linker to know the physical library name, the symbols however are only resolved at load time), while in Linux everything besides linking to the dynamic linker happen at load time.
Dynamic linking is in general referring to open manually the DLL file at runtime (such as using LoadLinrary()), in which case the burden is entirely on the programmer.
In shared library, such as .dll .dylib and .so, there is some information about symbol's name and address, like this:
------------------------------------
| symbol's name | symbol's address |
|----------------------------------|
| Foo | 0x12341234 |
| Bar | 0xabcdabcd |
------------------------------------
And the load function, such as LoadLibrary and dlopen, loads shared library and make it available to use.
GetProcAddress and dlsym find you symbol's address. For example:
HMODULE shared_lib = LoadLibrary("asdf.dll");
void *symbol = GetProcAddress("Foo");
// symbol is 0x12341234
In windows, there is .lib file to use .dll. When you link to this .lib file, you don't need to call LoadLibrary and GetProcAddress, and just use shared library's function as if they're "normal" functions. How can it work?
In fact, the .lib contains an import information. It's like that:
void *Foo; // please put the address of Foo there
void *Bar; // please put the address of Bar there
When the operating system loads your program (strictly speaking, your module), operating system performs LoadLibrary and GetProcAddress automatically.
And if you write code such as Foo();, compiler convert it into (*Foo)(); automatically. So you can use them as if they're "normal" functions.
All in all, what I want is to avoid duplicate .lib files in several .dlls since the generated .dlls will be used together in a .exe file and this makes global state duplicated in the .lib file duplicated.
Concrete Problem:
I have a project with 3 libraries and one .exe:
libDependentA.dll
libDependentB.dll
libIndependent.lib -- static library
The dependencies are as follow:
libDependentA.dll -- depends on -> libIndependent.lib
libDependentB.dll -- depends on -> libIndependent.lib
I have an .exe. Depends on both libDependentA.dll and libDependentB.dll
This makes two copies of my .lib into the .exe, which is what is causing problems.
Solutions?
Preferred solution. Don't know if possible, when googling no info.
Make a .dll out of the libIndependent.lib. How can I do this? I couldn't make a dll out of .lib files in cmake through add_libary(newdllfromlib SHARED). It has no source files dependencies and I tried to target_link_libraries into
newdllfromlib. Is there an easy way without using empty source files or other tricks?
Make libDependentB.dll depend only in libDependentA.dll and remove libIndependent.lib from libDependentB.dll. How can I do that in cmake? The problem here seems to be that cmake transitively propagates to all the other places libIndependent.lib, including the .exe, if it appears in target_link_libraries(libDependentA.... If I add LINK_PRIVATE, then, I cannot set up the configuration I want in cmake, because I must link libIndependent.lib again into the libDependentB.dll. Anyway to do it?
Duplication of symbols in .dlls from linked static libraries is what static libraries are supposed to do. If the static library libIndependent is 3rd party code, then the developers of libIndependent probably have some reason why they chosen the static library and not a dynamic library.
If you use symbols from libIndependent in both libDependentA and libDependentB and you do not have source code of libIndependent, the short answer is that you cannot do it directly. Static libraries are usually build with different set of flags/defines than shared libraries (most obvious is exporting the symbols dllexport / dllimport as you noted). You can export some libIndependent symbols from libDependentA by def file. But usually only symbols that are used in dll are present in the library (libDependentA) and you can run into various other problems.
However you can create a shared library wrapper libIndependentWrapper.dll, where you create a wrapper function for every function from libIndependent you need. The wrapper functions would have new header with dllexport / dllimport.
I have a framework (in C++) which is dependent on a few third party libraries. When I compile a static version of the library framework, no additional dependencies are needed, that is, the lib files of the third part libraries are not needed. When I compile the same framework as a DLL, additional dependencies are now needed otherwise I get linking errors. I can guess as to why this is happening but would like a concrete answer/explanation to understand what is happening.
EDIT: Just to clarify, I am developing a framework which can be compiled as a lib and as a dll and then used in a(n) (executable) project. When compiling the framework as a lib and using functions from a third party library, I don't need additional dependencies. However, a project that now uses the lib file (which is the framework) must include the 3rd party lib files. When I compile the framework as a dll it gives me linking errors unless I specify the 3rd part libraries the framework is technically dependent on. For example: I have a few classes that call functionality from within Ogre3D. These classes are compiled as a lib file. I don't need to link against OgreMain.lib when compiling a lib of the classes. On the other hand, when I am compiling a dll version of the same classes, I now need to link against OgreMain.lib
When you have a static library (a .lib file), which is just a collection of one or more object files (.obj), the linker just adds that code to yours in one executable. You can tell the linker to do this via a command line switch, an IDE configuration setting, or perhaps even a #pragma (specifics depend on your environment and compiler).
When you link in a DLL, you need to give the linker some code to call when you invoke one of the DLLs functions. Usually, this is done with a file of the same name as the .dll, save that it is a .lib. The code in that .lib is linked into your program the same way as described above, but when you call it, it loads the DLL (if not already loaded) and then invokes the proper function.
There are other ways to handle DLL linking (for instance, .def files or #using statements in .NET), but this seems to be what you're talking about.
Responding to your question clarification:
The issue is that a .lib is not a final product. It is just an aggregation of object code to be used later when a linker connects all your functions calls to function addresses.
A DLL, on the other hand, is a final product, and so the linker requires all functions and variables be connected to actual addresses.
I'm speaking a bit imprecisely, but you get the idea.
A static library can include other static libraries, providing a single lib to link
A DLL can include static libraries, providing a single DLL to link.
A DLL or static library with dependencies on other DLLs has no way to combine them so your executable must explicitly link to those other DLLs.
When you link to a LIB it adds all the symbols/functions you actually use to your executable. The ones you don't use won't get added. When you link to a dll - all the code from the external library gets loaded. If this additional code (code you don't use) depends on more external libraries you need to provide these as well.
One example: You want to use a ip class from a network library. The ip class does not depend on other libraries. Other functions in the network library depend on other external libraries. If you link the network library as a LIB you just link the ip class -> you don't need the other libraries since the other code wont get linked. When you use the DLL all code in the dll need to be instanciated -> so you will need to provide the other external libraries.
Building a DLL is more like building an application than a library. The difference between building an application and a DLL is knowledge of what might be called. In an application all symbols that are not used can be discarded in the build, but in a DLL you cannot strip symbols that are not used - that would be all of them...
You would find the same link problems in your static libraries if you where able to call all the symbols that the DLL links.
It's pointed out by this answer:
Failed to link mysql5.1.39\bin\libmySQL.dll
But I don't understand why,.dll is essentially the same as .lib except for there is only one copy of it used by different processes.
Does it have anything to do with the IDE?I'm using visual c++ 2008 express
UPDATE
Anyone know a free tool to convert .dll into .lib in windows?
You are wrong on two counts. Firstly, DLLs and LIBs (static libraries) are very different beasts. The LIB you are talking about (I think) is the export library, which is simply a list of names in the DLL. This library is typically produced when the DLL is compiled and is shipped with the DLL if the DLL is intended to be linked to by other developers.
To use a DLL with a modern IDE (I don't use VS) you typically include the matching .LIB (export library) in the project. At run-time you must make sure the DLL is available to your program - the simplest way to do this is to put the DLL in the same directory as the executable.
And secondly, DLLs can be used with C++.
DLL are specific windows executables which load on runtime. Their equivalent in Linux is the *.so .
If you want to understand what's the difference between a DLL and a static lib see here.
Main reason has probably something to do with dll-file being an output of the linker (link.exe). This command line utility doesnt know how to read dlls, only how to write them.
Actually sometimes .lib-files contain more than a list of functions. For example libpng.lib works as a standalone file, without any dll file.
In order to use a DLL in C/C++ you need to link with what is called an import lib. This is a small .lib that contains the names/ordinals the DLL exports in a format that the linker understands. Once this has been linked in, the resulting binary will be able to use the DLL. If you do not have the appropriate import lib, your C/C++ compiler should come with a tool to generate one from the DLL.
You can use the following analogy to understand the difference between a dll and a lib file.
the dll is like the source .cpp file while the lib is the header .h file.
While this is not exactly true, the DLL holds the executable code while the LIB file tells the linker how to glue the code together.
It is also possible (in some cases) to generate the lib from the dll. Basically, all you need to know to call a function is the function entry point in the dll, the number of parameters and the size of each parameters. Sending relevant information is then your own problem.