e.exe is linked against my custom static library, c.lib, which uses Win32 API defined in w.dll. w.dll is located in C:\Windows\System32 and its import library is w.lib, located in Windows SDK directory. Shell w.lib be listed as Additional Dependency in c.lib or e.exe project? (e.exe builds successfully in both cases.) What is the best practice and why? I guess e.exe should not know about w.lib.
c.lib is intended to be shared among a group of developers only (not to be shipped to customers).
TEST: I used VS2008 and dumpbin utility to test both cases and here are results:
Case 1: w.lib added as Additional Dependency in c.lib project.
dumpbin /archivemembers c.lib output lists both offsets in w.dll and .obj files from c.lib project as Archive members.
Case 2: w.lib not added as Additional Dependency in c.lib but in e.exe project:
This time, dumpbin output contains only .obj files of c.lib and the size of c.lib is smaller than in Case 1
(c.lib was added as Additional Dependency in w.exe project in both cases.)
NOTE: I used w.lib and w.dll here as fictional, generic names for Windows libraries but they could be e.g. Userenv.lib and Userenv.dll or Version.lib and Version.dll...
I think you're misunderstanding what creating an archive and an import archive does.
Creating an archive, as you've rightly surmised in comments, creates a unified file containing compiled .objs. Now, this can contain any code you like, including but not limited to dynamic calls to libraries. An import library is a library that contains an obj that exclusively makes such calls, the idea being that by importing it, your exe can find the appropriate symbols (they must be in the executable you create).
The process of creating c.lib from w.lib simply extracts w.lib's objects and appends them to the collection of objects in c.lib. In effect, c.lib becomes an import library + code.
Do I think you should do this? Not really - it might lead to confusion as to what e.exe depends on; I think you should explicitly make this visible rather than trying to hide it. That said, that's a recommendation only, not a rule.
Libraries are not linked, so any project that uses a .lib also needs its dependencies.
Basically the .lib are "copied" to your exe during linking.
If you want to avoid your users to explicity link againts w.lib, transforms c.lib in a dll, dlls are linked and you do not need their dependencies during build.
Related
I have a cross-language (C++ <-> FORTRAN) cyclic dependency that would be too much of a pain to eradicate, so I live with it. It only requires an occasional /FORCE with no linker dependencies on my C++ library. I have been doing this manually as required, but the new version of our product has eight configurations (and possibly more in future) and it is becoming more of a pain.
I could create 'Forced' configurations for each build in the Visual Studio configuration manager, or 'Forced' copies of my project(s). However, both these approaches are a bit of a maintenance headache - changes to the project have to be spread to all configurations / project copies.
Can anyone think of a method of quickly building my 'forced' configurations without have to flip-flop settings or maintaining a sync'd configuration just for that purpose?
As mentioned in the comments, if the DLL's are closely coupled it may make more sense just to combine them into a single DLL.
However, if two DLLs are still required, then one solution to this is to split one of the projects that currently builds a DLL into two - a static library project that also builds an import library and an exports file from a module definition file, and a second project that builds the DLL.
These two projects then bookend the construction of the other DLL.
For example, choosing to split the Fortran DLL, as I am more familiar with its project system:
Create a module definition file, with the same base name as will be used for the Fortran DLL, that lists in its exports section all the symbols that the Fortran DLL will export.
Create a Fortran static library project with a name different from that of the final Fortran DLL, that is configured to compile all the Fortran sources. In the project properties, as a custom build step add an additional invocation of the librarian along the lines of lib /DEF:xxx.def /OUT:xxx.dll /MACHINE:x86 (where xxx is the base name that will be used for the Fortran DLL - prepend configuration names as path suffixes as appropriate, vary the machine option as appropriate). Building this Fortran static library project will now generate two libraries - one with the object code (named after the project) and one that is the import library (named after the DLL) and an exports file (also named after the DLL).
[Note that with this approach the import library doesn't actually depend on the object code generated when the Fortran source files are compiled - use of the custom build step of the static library project is just a convenience. If you don't already have a module definition file for the Fortran DLL, an alternative approach is to instead supply the individual object files in the invocation of the librarian in the custom build step, and let the librarian determine the exports from any directives in the source files. However, I prefer module definition files to in source directives.]
Create a C++ project that holds all the C++ sources and builds the C++ DLL. That project should depend on the Fortran static library project and link against the import library generated by the Fortran static library project.
Create a Fortran DLL project, with the same base name as the Fortran DLL, that holds a dummy Fortran source file which only has comments inside it (this is just to avoid the build system getting confused). This DLL project should depend on the C++ DLL project, and link against the import library generated by the C++ project. In the project properties, under Linker > Input > Additional Dependencies, also add in the static library (not the import library!) and exports file generated by the Fortran static library project.
Other direct clients of the Fortran DLL should link against the import library generated by the custom build step of the Fortran static library project.
Variations on this approach are possible.
All in all, what I want is to avoid duplicate .lib files in several .dlls since the generated .dlls will be used together in a .exe file and this makes global state duplicated in the .lib file duplicated.
Concrete Problem:
I have a project with 3 libraries and one .exe:
libDependentA.dll
libDependentB.dll
libIndependent.lib -- static library
The dependencies are as follow:
libDependentA.dll -- depends on -> libIndependent.lib
libDependentB.dll -- depends on -> libIndependent.lib
I have an .exe. Depends on both libDependentA.dll and libDependentB.dll
This makes two copies of my .lib into the .exe, which is what is causing problems.
Solutions?
Preferred solution. Don't know if possible, when googling no info.
Make a .dll out of the libIndependent.lib. How can I do this? I couldn't make a dll out of .lib files in cmake through add_libary(newdllfromlib SHARED). It has no source files dependencies and I tried to target_link_libraries into
newdllfromlib. Is there an easy way without using empty source files or other tricks?
Make libDependentB.dll depend only in libDependentA.dll and remove libIndependent.lib from libDependentB.dll. How can I do that in cmake? The problem here seems to be that cmake transitively propagates to all the other places libIndependent.lib, including the .exe, if it appears in target_link_libraries(libDependentA.... If I add LINK_PRIVATE, then, I cannot set up the configuration I want in cmake, because I must link libIndependent.lib again into the libDependentB.dll. Anyway to do it?
Duplication of symbols in .dlls from linked static libraries is what static libraries are supposed to do. If the static library libIndependent is 3rd party code, then the developers of libIndependent probably have some reason why they chosen the static library and not a dynamic library.
If you use symbols from libIndependent in both libDependentA and libDependentB and you do not have source code of libIndependent, the short answer is that you cannot do it directly. Static libraries are usually build with different set of flags/defines than shared libraries (most obvious is exporting the symbols dllexport / dllimport as you noted). You can export some libIndependent symbols from libDependentA by def file. But usually only symbols that are used in dll are present in the library (libDependentA) and you can run into various other problems.
However you can create a shared library wrapper libIndependentWrapper.dll, where you create a wrapper function for every function from libIndependent you need. The wrapper functions would have new header with dllexport / dllimport.
I've got a huge application here called HugeApp, it needs different libraries (which I've coded) and some of these libraries might need dependencies (other libs coming from the internet or ad hoc lib developped here).
I was wondering, if it was feasible and/or a good idea to hide some of these dependencies from HugeApp.
Let's say that you make a library in charge of doing the encrypted communication on a system, does the top application care and/or needs to know that there is some encryption libraries that are needed for this part (comms) of the system? It might be implementation specific... or not...
Thank you
There's no need for it to know, if you build those libraries as external DLL's then the external libraries are the only thing that care about the dependency. If you add a reference to the pre-built DLL then HugeApp doesn't need to know about the dependencies of the library (as long as they are either present in the library or the appropriate DLL or lib file is present so that your dll can make use of it). If anything your library can be another project altogether and you can include a reference to that in which case the project of your HugeApp only cares about that main reference and the other project will handle everything else.
If you enable /OPT:REF in linker optimizations it you will list which (if any) libraries that have no functions or data used by the project during link time. You can then remove them from the dependency list and link line in project settings. This will reduce the chance of removing a static library that is a dependency of another static library if any are present/used in your VS solution.
I have a framework (in C++) which is dependent on a few third party libraries. When I compile a static version of the library framework, no additional dependencies are needed, that is, the lib files of the third part libraries are not needed. When I compile the same framework as a DLL, additional dependencies are now needed otherwise I get linking errors. I can guess as to why this is happening but would like a concrete answer/explanation to understand what is happening.
EDIT: Just to clarify, I am developing a framework which can be compiled as a lib and as a dll and then used in a(n) (executable) project. When compiling the framework as a lib and using functions from a third party library, I don't need additional dependencies. However, a project that now uses the lib file (which is the framework) must include the 3rd party lib files. When I compile the framework as a dll it gives me linking errors unless I specify the 3rd part libraries the framework is technically dependent on. For example: I have a few classes that call functionality from within Ogre3D. These classes are compiled as a lib file. I don't need to link against OgreMain.lib when compiling a lib of the classes. On the other hand, when I am compiling a dll version of the same classes, I now need to link against OgreMain.lib
When you have a static library (a .lib file), which is just a collection of one or more object files (.obj), the linker just adds that code to yours in one executable. You can tell the linker to do this via a command line switch, an IDE configuration setting, or perhaps even a #pragma (specifics depend on your environment and compiler).
When you link in a DLL, you need to give the linker some code to call when you invoke one of the DLLs functions. Usually, this is done with a file of the same name as the .dll, save that it is a .lib. The code in that .lib is linked into your program the same way as described above, but when you call it, it loads the DLL (if not already loaded) and then invokes the proper function.
There are other ways to handle DLL linking (for instance, .def files or #using statements in .NET), but this seems to be what you're talking about.
Responding to your question clarification:
The issue is that a .lib is not a final product. It is just an aggregation of object code to be used later when a linker connects all your functions calls to function addresses.
A DLL, on the other hand, is a final product, and so the linker requires all functions and variables be connected to actual addresses.
I'm speaking a bit imprecisely, but you get the idea.
A static library can include other static libraries, providing a single lib to link
A DLL can include static libraries, providing a single DLL to link.
A DLL or static library with dependencies on other DLLs has no way to combine them so your executable must explicitly link to those other DLLs.
When you link to a LIB it adds all the symbols/functions you actually use to your executable. The ones you don't use won't get added. When you link to a dll - all the code from the external library gets loaded. If this additional code (code you don't use) depends on more external libraries you need to provide these as well.
One example: You want to use a ip class from a network library. The ip class does not depend on other libraries. Other functions in the network library depend on other external libraries. If you link the network library as a LIB you just link the ip class -> you don't need the other libraries since the other code wont get linked. When you use the DLL all code in the dll need to be instanciated -> so you will need to provide the other external libraries.
Building a DLL is more like building an application than a library. The difference between building an application and a DLL is knowledge of what might be called. In an application all symbols that are not used can be discarded in the build, but in a DLL you cannot strip symbols that are not used - that would be all of them...
You would find the same link problems in your static libraries if you where able to call all the symbols that the DLL links.
I know this may seem quite basic to geeks. But I want to make it crystal clear.
When I want to use a Win32 DLL, usually I just call the APIs like LoadLibrary() and GetProcAdderss(). But recently, I am developing with DirectX9, and I need to add d3d9.lib, d3dx9.lib, etc files.
I have heard enough that LIB is for static linking and DLL is for dynamic linking.
So my current understanding is that LIB contains the implementation of the methods and is statically linked at link time as part of the final EXE file. While DLL is dynamic loaded at runtime and is not part of the final EXE file.
But sometimes, there're some LIB files coming with the DLL files, so:
What are these LIB files for?
How do they achieve what they are meant for?
Is there any tools that can let me inspect the internals of these LIB files?
Update 1
After checking wikipedia, I remember that these LIB files are called import library.
But I am wondering how it works with my main application and the DLLs to be dynamically loaded.
Update 2
Just as RBerteig said, there're some stub code in the LIB files born with the DLLs. So the calling sequence should be like this:
My main application --> stub in the LIB --> real target DLL
So what information should be contained in these LIBs? I could think of the following:
The LIB file should contain the fullpath of the corresponding DLL; So the DLL could be loaded by the runtime.
The relative address (or file offset?) of each DLL export method's entry point should be encoded in the stub; So correct jumps/method calls could be made.
Am I right on this? Is there something more?
BTW: Is there any tool that can inspect an import library? If I can see it, there'll be no more doubts.
Linking to a DLL file can occur implicitly at compile link time, or explicitly at run time. Either way, the DLL ends up loaded into the processes memory space, and all of its exported entry points are available to the application.
If used explicitly at run time, you use LoadLibrary() and GetProcAddress() to manually load the DLL and get pointers to the functions you need to call.
If linked implicitly when the program is built, then stubs for each DLL export used by the program get linked in to the program from an import library, and those stubs get updated as the EXE and the DLL are loaded when the process launches. (Yes, I've simplified more than a little here...)
Those stubs need to come from somewhere, and in the Microsoft tool chain they come from a special form of .LIB file called an import library. The required .LIB is usually built at the same time as the DLL, and contains a stub for each function exported from the DLL.
Confusingly, a static version of the same library would also be shipped as a .LIB file. There is no trivial way to tell them apart, except that LIBs that are import libraries for DLLs will usually be smaller (often much smaller) than the matching static LIB would be.
If you use the GCC toolchain, incidentally, you don't actually need import libraries to match your DLLs. The version of the Gnu linker ported to Windows understands DLLs directly, and can synthesize most any required stubs on the fly.
Update
If you just can't resist knowing where all the nuts and bolts really are and what is really going on, there is always something at MSDN to help. Matt Pietrek's article An In-Depth Look into the Win32 Portable Executable File Format is a very complete overview of the format of the EXE file and how it gets loaded and run. Its even been updated to cover .NET and more since it originally appeared in MSDN Magazine ca. 2002.
Also, it can be helpful to know how to learn exactly what DLLs are used by a program. The tool for that is Dependency Walker, aka depends.exe. A version of it is included with Visual Studio, but the latest version is available from its author at http://www.dependencywalker.com/. It can identify all of the DLLs that were specified at link time (both early load and delay load) and it can also run the program and watch for any additional DLLs it loads at run time.
Update 2
I've reworded some of the earlier text to clarify it on re-reading, and to use the terms of art implicit and explicit linking for consistency with MSDN.
So, we have three ways that library functions might be made available to be used by a program. The obvious follow up question is then: "How to I choose which way?"
Static linking is how the bulk of the program itself is linked. All of your object files are listed, and get collected together in to the EXE file by the linker. Along the way, the linker takes care of minor chores like fixing up references to global symbols so that your modules can call each other's functions. Libraries can also be statically linked. The object files that make up the library are collected together by a librarian in a .LIB file which the linker searches for modules containing symbols that are needed. One effect of static linking is that only those modules from the library that are used by the program are linked to it; other modules are ignored. For instance, the traditional C math library includes many trigonometry functions. But if you link against it and use cos(), you don't end up with a copy of the code for sin() or tan() unless you also called those functions. For large libraries with a rich set of features, this selective inclusion of modules is important. On many platforms such as embedded systems, the total size of code available for use in the library can be large compared to the space available to store an executable in the device. Without selective inclusion, it would be harder to manage the details of building programs for those platforms.
However, having a copy of the same library in every program running creates a burden on a system that normally runs lots of processes. With the right kind of virtual memory system, pages of memory that have identical content need only exist once in the system, but can be used by many processes. This creates a benefit for increasing the chances that the pages containing code are likely to be identical to some page in as many other running processes as possible. But, if programs statically link to the runtime library, then each has a different mix of functions each laid out in that processes memory map at different locations, and there aren't many sharable code pages unless it is a program that all by itself is run in more than process. So the idea of a DLL gained another, major, advantage.
A DLL for a library contains all of its functions, ready for use by any client program. If many programs load that DLL, they can all share its code pages. Everybody wins. (Well, until you update a DLL with new version, but that isn't part of this story. Google DLL Hell for that side of the tale.)
So the first big choice to make when planning a new project is between dynamic and static linkage. With static linkage, you have fewer files to install, and you are immune from third parties updating a DLL you use. However, your program is larger, and it isn't quite as good citizen of the Windows ecosystem. With dynamic linkage, you have more files to install, you might have issues with a third party updating a DLL you use, but you are generally being friendlier to other processes on the system.
A big advantage of a DLL is that it can be loaded and used without recompiling or even relinking the main program. This can allow a third party library provider (think Microsoft and the C runtime, for example) to fix a bug in their library and distribute it. Once an end user installs the updated DLL, they immediately get the benefit of that bug fix in all programs that use that DLL. (Unless it breaks things. See DLL Hell.)
The other advantage comes from the distinction between implicit and explicit loading. If you go to the extra effort of explicit loading, then the DLL might not even have existed when the program was written and published. This allows for extension mechanisms that can discover and load plugins, for instance.
These .LIB import library files are used in the following project property, Linker->Input->Additional Dependencies, when building a bunch of dll's that need additional information at link time which is supplied by the import library .LIB files. In the example below to not get linker errors I need to reference to dll's A,B,C, and D through their lib files. (note for the linker to find these files you may need to include their deployment path in Linker->General->Additional Library Directories else you will get a build error about being unable to find any of the provided lib files.)
If your solution is building all dynamic libraries you may have been able to avoid this explicit dependency specification by relying instead on the reference flags exposed under the Common Properties->Framework and References dialog. These flags appear to automatically do the linking on your behalf using the *.lib files.
This however is as it says a Common Properties, which is not configuration or platform specific. If you need to support a mixed build scenario as in our application we had a build configuration to render a static build and a special configuration that built a constrained build of a subset of assemblies that were deployed as dynamic libraries. I had used the Use Library Dependency Inputs and Link Library Dependencies flags set to true under various cases to get things to build and later realizing to simplify things but when introducing my code to the static builds I introduced a ton of linker warnings and the build was incredibly slow for the static builds. I wound up introducing a bunch of these sort of warnings...
warning LNK4006: "bool __cdecl XXX::YYY() already defined in CoreLibrary.lib(JSource.obj); second definition ignored D.lib(JSource.obj)
And I wound up using the manual specification of Additional Dependencies to satisfy the linker for the dynamic builds while keeping the static builders happy by not using a common property that slowed them down. When I deploy the dynamic subset build I only deploy the dll files as these lib files are only used at link time, not at runtime.
Here are some related MSDN topics to answer my question:
Linking an Executable to a DLL
Linking Implicitly
Determining Which Linking Method to Use
Building an Import Library and Export File
There are three kinds of libraries: static, shared and dynamically loaded libraries.
The static libraries are linked with the code at the linking phase, so they are actually in the executable, unlike the shared library, which has only stubs (symbols) to look for in the shared library file, which is loaded at run time before the main function gets called.
The dynamically loaded ones are much like the shared libraries, except they are loaded when and if the need arises by the code you've written.
In my mind, there are two method to link dll to exe.
Use dll and the import library (.lib file) implicitly
Use functions like loadlibrary() explicitly