I don't understand this paragraph :
Exporting functions in a .def file gives you control over the export ordinals. When you add an exported function to your DLL, you can assign it a higher ordinal value than any other exported function. When you do this, applications that use implicit linking do not have to relink with the import library that contains the new function. This is very convenient if you are designing a DLL for use by many applications because you can add new functionality and also ensure that it continues to work correctly with the applications that already rely on it. For example, the MFC DLLs are built by using .def files.
Why application doesn't have to relink with the import library in case of the usage of a .def file instead of __declspec(dllexport) in the case of a function adding into the dll ?
cf https://learn.microsoft.com/en-us/cpp/build/determining-which-exporting-method-to-use
That is because of some specifics of MSFT implementation of shared objects (or DLLs). In Microsoft world, in order to import function into your process, you need not only the shared code itself (.dll), but you also need the special 'import' library - the .lib file. This file is statically linked into your application (as it is a static library). This library provides 'glue' between function names and function ordinals.
Normally, every time you release a new version of DLL, all applications which use it will have to be relinked with the new, accompanying version of static import library (.lib) to be able to use this new DLL library. This is due to the fact that function ordinals are generally no longer valid after you have created the new library. However, if you are using .def file, you can assign ordinals manually, and ensure the ordinals remain the same for previously available functions - and thus .lib file will still be valid.
Ok, if you have a .def file you can use it to create an import library.
I.e. mydll.lib for MS VC++ or mylib-dll.a for GCC
Compilers and linkers prefer their own binary format import libraries, usually not compatible with each-other. This is especially does mater when you'r DLL is written on C/C++ but your program is written on something else like Ada/FORTRAN/Object Pascal etc or vise versa. So .def files can be used to create a compatible import library.
Paragraph telling you a way to hide some functions from import library, with manual editing .DEF file and instruct linker to hide some functions.
Related
Under Windows, when I compile C/C++ code in a DLL project in MSVC I am getting 2 files:
MyDll.dll
MyDll.lib
where as far as I understand MyDll.lib contains some kind of pointers table indicating functions locations in the dll. When using this dll, say in an exe file, MyDll.lib is embedded into the exe file during linkage so in runtime it "knows" where the functions are located in MyDll.dll and can use them.
But if I compile the same code under Linux I am getting only one file MySo.so without MySo.a (the equivalent to lib file in Linux) so how does an executable file under Linux knows where the functions are located in MySo.so if nothing is embedded into it during linking?
The MSVC linker can link together object files (.obj) and object libraries (.lib) to produce an .EXE or a .DLL.
To link with a DLL, the process in MSVC is to use a so-called import library (.LIB) that acts as a glue between the C function names and the DLL's export table (in a DLL a function can be exported by name or by ordinal - the latter was often used for undocumented APIs).
However, in most cases the DLL export table has all the function names and thus the import library (.LIB) contains largely redundant information ("import function ABC -> exported function ABC", etc).
It is even possible to generate a .LIB from an existing .DLL.
Linkers on other platforms don't have this "feature" and can link with dynamic libraries directly.
On Linux, the linker (not the dynamic linker) searches through the shared libraries specified at link time and creates references to them inside the executable. When the dynamic linker loads these executables it loads the shared libraries they require into memory and resolves the symbols, which allows the binaries to be run.
MySo.a, if created, would actually include the symbols to be linked directly into the binary instead of the "symbol lookup tables" used on Windows.
rustyx's answer explains the process on Windows more thoroughly than I can; it's been a long time since I've used Windows.
The difference you are seeing is more of an implementation detail - under the hood both Linux and Windows work similarly - you code calls a stub function which is statically linked in your executable and this stub then loads DLL/shlib if necessary (in case of delayed loading, otherwise library is loaded when program starts) and (on first call) resolves symbol via GetProcAddress/dlsym.
The only difference is that on Linux the these stub functions (which are called PLT stubs) are generated dynamically when you link your app with dynamic library (library contains enough information to generate them), whereas on Windows they are instead generated when DLL itself is created, in a separate .lib file.
The two approaches are so similar that it's actually possible to mimic Windows import libraries on Linux (see Implib.so project).
On Linux, you pass MySo.so to the linker and it is able to extract only what is needed for the link phase, putting in a reference that MySo.so is needed at run time.
.dll or .so are shared libs (linked in runtime), while .a and .lib is a static library (linked in compile time). This is no difference between Windows and Linux.
The difference is, how are they handled. Note: the difference is only in the customs, how are they used. It wouldn't be too hard to make Linux builds on the Windows way and vice versa, except that practically no one does this.
If we use a dll, or we call a function even from our own binary, there is a simple and clear way. For example, in C, we see that:
int example(int x) {
...do_something...
}
int ret = example(42);
However, on the asm level, there could be many differences. For example, on x86, a call opcode is executed, and the 42 is given on the stack. Or in some registers. Or anywhere. No one knows that before writing the dll, how it will be used. Or how the projects will want to use it, possible written with a compiler (or in a language!) which doesn't even exist now (or is it unknown for the developers of the dll).
For example, by default, both C and Pascal puts the arguments (and gets the return values) from the stack - but they are doing it in different order. You can also exchange arguments between your functions in the registers by some - compiler-dependent - optimization.
As you see correctly, the Windows custom is that building a dll, we also create a minimal .a/.lib with it. This minimal static library is only a wrapper, the symbols (functions) of that dll are reached through it. This makes the required asm-level calling conversions.
Its advantage is the compatibility. Its disadvantage is that if you have only a .dll, you can have a hard time to figure out, how its functions want to be called. This makes the usage of dlls a hacking task, if the developer of the dll does not give you the .a. Thus, it serves mainly closedness purposes, for example so is it easier to get extra cash for the SDKs.
Its another disadvantage is than even if you use a dynamical library, you need to compile this little wrapper statically.
In Linux, the binary interface of the dlls is standard and follows the C convention. Thus, no .a is required and there is binary compatibility between the shared libs, in exchange we don't have the advantages of the microsoft custom.
Below is an excerpt from link1.
Microsoft introduced __export in the 16-bit compiler version of Visual
C++ to allow the compiler to generate the export names automatically
and place them in a .lib file. This .lib file can then be used just
like a static .lib to link with a DLL. In newer compiler versions, you
can export data, functions, classes, or class member functions from a
DLL using the __declspec(dllexport) keyword. __declspec(dllexport)
adds the export directive to the object file so you do not need to use
a .def file.
I understand the above paragraph to an extent but not very well.
Below is an excerpt from link2.
When building the DLL, the linker uses the .def file to create an
export (.exp) file and an import library (.lib) file. The linker then
uses the export file to build the DLL file. Executables that
implicitly link to the DLL link to the import library when they are
built.
Now, this makes me confused and made me ask the below questions:
Could anybody, in simple words, tell me what the term exporting
really means? I believe this is making an object accessible from one
piece of the code to other - but hey !!
When building projects with old libraries, I see .def file in majority
of them. But the latest compilers automatically exports objects. Would
the presence of the .def file cause any conflict when converting a older
version visual studio project to the newer one?
What is the use of the .lib(the so called import file) after the generation
of the .dll. Can it be safely deleted?
ARRGGGH !! What is the difference between a static library(.lib) and import library(.lib)? Blunder, huh? But still !!
Is the windows specific phenomenon? I believe it is not. What is the Linux counterpart of the so called import file?
Please feel free to rephrase the question if it is not already lucid.
tell me what the term exporting really means?
It simply means telling the linker that it needs to put an entry into the DLL's export table. The operating system loader uses it later to glue code in different modules together at runtime.
I see .def file in majority of them
Could be very old projects. Or it was just never started as a project that was meant to create a separate module. Like a static library so the source code doesn't have the __declspec attributes. Cross-platform libraries are pretty likely to fit that bill. The C and C++ language specifications still don't have a way to create modules in a standardized way. Everybody does it, nobody does it the same way. Massive time drain.
What is the use of the .lib(the so called import file)
It is necessary in the project that uses the DLL. The linker needs to know that the identifier lives in another building and can't be resolved at link time. It puts an entry in another table that the operating system loader uses, the import table. It is a very simple file, it just list the names of the exported identifiers. Theoretically the DLL itself could be used by the linker to figure this out. Practically that doesn't work because the exported name doesn't have to match the actual name.
What is the difference between a static library(.lib) and import library(.lib)
A static library contains code that is linked into the project that uses the library. An import library does not contain code, just a hint that the code is available elsewhere.
Is the windows specific phenomenon?
Roughly, yes. The Unixes have the same concept but implement it very differently.
I don't know how to do the following:
I'm using MS Visual C++ 6.0
I have a Win32 DLL project which is compilable.
I have another project, this time a Win32 Console project which uses
the DLL by including it's header file and linking the .lib file of
the DLL.
Now I want to have another project, similar to the second BUT without using the header file and the lib file.
Is that possible? Everywhere I read you need either dll+lib+h or dll+h. If thought if you know the interfaces, a DLL file is sufficient?
Btw, by "using a DLL" I mean, using the Classes and Functions defined in the DLL.
It is possible if you just have plain "extern C" functions. If this is the case the approach could be loading the dll with LoadLibrary, and then import each function with GetProcAddress, of course you need to know the function signature to create a properly declared function pointer. Using classes per contrary is almost impossible.
If your DLL contains classes, there are good chances that it is a COM component.
If this is the case, the #import directive (that you use like #include) builds some temporary include files containing the interface details. You should use COM to access your objects.
Otherwise, if you have a 'plain' DLL with C++ classes, you could access the exported symbols using linker: instruct it to dump the map (see here), to know the mangled names. But I don't think that's possible to build manually the interface...
I have a few NT imports I want to use in my program, problem is I am unable to use them without going though the lengthly process of downloading and setting up the WDK to use just two functions. I would also prefer not to use GetModuleHandle and GetProcAddress.
I know in VB6 you can manually define imported functions from dlls like this:
Private Declare Function NtFunction Lib "ntdll.dll" (function arguments) As type
Is there something similar I can do with C++ in Visual Studio without having all the headers/libs?
You say you don't want to use GetProcAddress, but that's exactly what VB6 Declare Function (and .NET p/invoke) does.
You do need a complete prototype, often you can recreate enough of the header file from documentation.
The import library is a little more difficult, but there are tools to create import libraries from the .DLL.
If you create a .DEF file, you can use the LIB.EXE tool that comes with Visual C++ (and also available as a free download as part of the Windows SDK), see Building an Import Library
Here is some more information.
mingw comes with a tool for mostly automating creation of the .DEF file: http://www.mingw.org/wiki/CreateImportLibraries (the import library it creates is only usable with gcc, but the .DEF file is useful for making a VC++ import library as described above).
Here's a another tool that might help: http://www.codeproject.com/KB/tips/ImpDef.aspx
I know this may seem quite basic to geeks. But I want to make it crystal clear.
When I want to use a Win32 DLL, usually I just call the APIs like LoadLibrary() and GetProcAdderss(). But recently, I am developing with DirectX9, and I need to add d3d9.lib, d3dx9.lib, etc files.
I have heard enough that LIB is for static linking and DLL is for dynamic linking.
So my current understanding is that LIB contains the implementation of the methods and is statically linked at link time as part of the final EXE file. While DLL is dynamic loaded at runtime and is not part of the final EXE file.
But sometimes, there're some LIB files coming with the DLL files, so:
What are these LIB files for?
How do they achieve what they are meant for?
Is there any tools that can let me inspect the internals of these LIB files?
Update 1
After checking wikipedia, I remember that these LIB files are called import library.
But I am wondering how it works with my main application and the DLLs to be dynamically loaded.
Update 2
Just as RBerteig said, there're some stub code in the LIB files born with the DLLs. So the calling sequence should be like this:
My main application --> stub in the LIB --> real target DLL
So what information should be contained in these LIBs? I could think of the following:
The LIB file should contain the fullpath of the corresponding DLL; So the DLL could be loaded by the runtime.
The relative address (or file offset?) of each DLL export method's entry point should be encoded in the stub; So correct jumps/method calls could be made.
Am I right on this? Is there something more?
BTW: Is there any tool that can inspect an import library? If I can see it, there'll be no more doubts.
Linking to a DLL file can occur implicitly at compile link time, or explicitly at run time. Either way, the DLL ends up loaded into the processes memory space, and all of its exported entry points are available to the application.
If used explicitly at run time, you use LoadLibrary() and GetProcAddress() to manually load the DLL and get pointers to the functions you need to call.
If linked implicitly when the program is built, then stubs for each DLL export used by the program get linked in to the program from an import library, and those stubs get updated as the EXE and the DLL are loaded when the process launches. (Yes, I've simplified more than a little here...)
Those stubs need to come from somewhere, and in the Microsoft tool chain they come from a special form of .LIB file called an import library. The required .LIB is usually built at the same time as the DLL, and contains a stub for each function exported from the DLL.
Confusingly, a static version of the same library would also be shipped as a .LIB file. There is no trivial way to tell them apart, except that LIBs that are import libraries for DLLs will usually be smaller (often much smaller) than the matching static LIB would be.
If you use the GCC toolchain, incidentally, you don't actually need import libraries to match your DLLs. The version of the Gnu linker ported to Windows understands DLLs directly, and can synthesize most any required stubs on the fly.
Update
If you just can't resist knowing where all the nuts and bolts really are and what is really going on, there is always something at MSDN to help. Matt Pietrek's article An In-Depth Look into the Win32 Portable Executable File Format is a very complete overview of the format of the EXE file and how it gets loaded and run. Its even been updated to cover .NET and more since it originally appeared in MSDN Magazine ca. 2002.
Also, it can be helpful to know how to learn exactly what DLLs are used by a program. The tool for that is Dependency Walker, aka depends.exe. A version of it is included with Visual Studio, but the latest version is available from its author at http://www.dependencywalker.com/. It can identify all of the DLLs that were specified at link time (both early load and delay load) and it can also run the program and watch for any additional DLLs it loads at run time.
Update 2
I've reworded some of the earlier text to clarify it on re-reading, and to use the terms of art implicit and explicit linking for consistency with MSDN.
So, we have three ways that library functions might be made available to be used by a program. The obvious follow up question is then: "How to I choose which way?"
Static linking is how the bulk of the program itself is linked. All of your object files are listed, and get collected together in to the EXE file by the linker. Along the way, the linker takes care of minor chores like fixing up references to global symbols so that your modules can call each other's functions. Libraries can also be statically linked. The object files that make up the library are collected together by a librarian in a .LIB file which the linker searches for modules containing symbols that are needed. One effect of static linking is that only those modules from the library that are used by the program are linked to it; other modules are ignored. For instance, the traditional C math library includes many trigonometry functions. But if you link against it and use cos(), you don't end up with a copy of the code for sin() or tan() unless you also called those functions. For large libraries with a rich set of features, this selective inclusion of modules is important. On many platforms such as embedded systems, the total size of code available for use in the library can be large compared to the space available to store an executable in the device. Without selective inclusion, it would be harder to manage the details of building programs for those platforms.
However, having a copy of the same library in every program running creates a burden on a system that normally runs lots of processes. With the right kind of virtual memory system, pages of memory that have identical content need only exist once in the system, but can be used by many processes. This creates a benefit for increasing the chances that the pages containing code are likely to be identical to some page in as many other running processes as possible. But, if programs statically link to the runtime library, then each has a different mix of functions each laid out in that processes memory map at different locations, and there aren't many sharable code pages unless it is a program that all by itself is run in more than process. So the idea of a DLL gained another, major, advantage.
A DLL for a library contains all of its functions, ready for use by any client program. If many programs load that DLL, they can all share its code pages. Everybody wins. (Well, until you update a DLL with new version, but that isn't part of this story. Google DLL Hell for that side of the tale.)
So the first big choice to make when planning a new project is between dynamic and static linkage. With static linkage, you have fewer files to install, and you are immune from third parties updating a DLL you use. However, your program is larger, and it isn't quite as good citizen of the Windows ecosystem. With dynamic linkage, you have more files to install, you might have issues with a third party updating a DLL you use, but you are generally being friendlier to other processes on the system.
A big advantage of a DLL is that it can be loaded and used without recompiling or even relinking the main program. This can allow a third party library provider (think Microsoft and the C runtime, for example) to fix a bug in their library and distribute it. Once an end user installs the updated DLL, they immediately get the benefit of that bug fix in all programs that use that DLL. (Unless it breaks things. See DLL Hell.)
The other advantage comes from the distinction between implicit and explicit loading. If you go to the extra effort of explicit loading, then the DLL might not even have existed when the program was written and published. This allows for extension mechanisms that can discover and load plugins, for instance.
These .LIB import library files are used in the following project property, Linker->Input->Additional Dependencies, when building a bunch of dll's that need additional information at link time which is supplied by the import library .LIB files. In the example below to not get linker errors I need to reference to dll's A,B,C, and D through their lib files. (note for the linker to find these files you may need to include their deployment path in Linker->General->Additional Library Directories else you will get a build error about being unable to find any of the provided lib files.)
If your solution is building all dynamic libraries you may have been able to avoid this explicit dependency specification by relying instead on the reference flags exposed under the Common Properties->Framework and References dialog. These flags appear to automatically do the linking on your behalf using the *.lib files.
This however is as it says a Common Properties, which is not configuration or platform specific. If you need to support a mixed build scenario as in our application we had a build configuration to render a static build and a special configuration that built a constrained build of a subset of assemblies that were deployed as dynamic libraries. I had used the Use Library Dependency Inputs and Link Library Dependencies flags set to true under various cases to get things to build and later realizing to simplify things but when introducing my code to the static builds I introduced a ton of linker warnings and the build was incredibly slow for the static builds. I wound up introducing a bunch of these sort of warnings...
warning LNK4006: "bool __cdecl XXX::YYY() already defined in CoreLibrary.lib(JSource.obj); second definition ignored D.lib(JSource.obj)
And I wound up using the manual specification of Additional Dependencies to satisfy the linker for the dynamic builds while keeping the static builders happy by not using a common property that slowed them down. When I deploy the dynamic subset build I only deploy the dll files as these lib files are only used at link time, not at runtime.
Here are some related MSDN topics to answer my question:
Linking an Executable to a DLL
Linking Implicitly
Determining Which Linking Method to Use
Building an Import Library and Export File
There are three kinds of libraries: static, shared and dynamically loaded libraries.
The static libraries are linked with the code at the linking phase, so they are actually in the executable, unlike the shared library, which has only stubs (symbols) to look for in the shared library file, which is loaded at run time before the main function gets called.
The dynamically loaded ones are much like the shared libraries, except they are loaded when and if the need arises by the code you've written.
In my mind, there are two method to link dll to exe.
Use dll and the import library (.lib file) implicitly
Use functions like loadlibrary() explicitly