It's common practice to strip a symbol table from a dynamic library (.dll on Windows, .dylib on OSX, and .so on Linux/Solaris/BSD). This makes sense because it drastically reduces the file size of the library, often more than 75 percent.
However, this one question's been bugging me: A stripped library has no symbol table. If I write an executable that references a function in this library, how does the operating system's dynamic linker know where to locate the section of code in the stripped library when there's no symbol table to provide this information?
This question comprises both the situation where the library was stripped before the executable was linked at compile-time and the situation where the library was stripped after the executable was linked at compile-time.
The symbols that are stripped when you run strip are the debugging symbols, not the names of actual exported symbols.
The dynamic symbols, the ones that the linker searches for, are still there, and can be listed by using the -D (Lists dynamic symbols) argument.
Related
Referring to this answer: https://stackoverflow.com/a/6264256/5324086,
I found that a linker has even more functionality than just managing absolute addresses for object file symbols.
What does the library produced by linker contain? Is it something other than ... say a C Standard library?
Why does the linker even need to produce a library?
The exact details depend on the type of library (you can search for shared library formats) but the basic components will include the compiled code, plus a symbol table that tells the linker which address corresponds to each name. Note that this is very similar to an object file. Static libraries are basically archives of object files and the compiler links them in a similar way. With dynamic libraries, the OS can look this up whenever it loads a program, and link the symbols then. They won't generally have the same absolute addresses in every program's address space, so these addresses will be relative to where the OS loads the library.
The C standard library (MSVC runtime on Windows) is an example of a library.
Static libraries are just a collection of object files. You can think of them as a tar file containing all the relevant .a files (or, on Windows, as a zip file containing obj files). The linking part of the linker is not involved here (in facts traditionally static libraries on Unix systems are done with the ar utility, which is somehow related to tar). They are completely resolved at compile time, and they are simply used as a way to avoid rebuilding all the time stuff that is long to build or has complex build procedures.
Dynamic libraries are a different beast. They are fully fledged executables that can be loaded by other processes, so the regular linker is needed for the same reasons it is used in normal executables. Instead of providing just a single entrypoint, they export a full symbols table that is used by the loader (or "runtime linker") to allow the host program to locate the required procedures. Generally they also contain relocation information to allow loading at any address in the target address space (or they are compiled in position independent code for this same reason).
I am trying to link a number of dynamic libraries into an application and running into problems with g++.
Consider:
libA.so
libB.so depends on libA.so
libC.so depends on libB.so
Application D depends directly on libC.so
If I try to link application D just to libC.so, I get unresolved symbols for the symbols in A and B. I feel as if the compiler should be able to figure it out, and when I use the intel compiler, it does. G++, however, can't figure out the linking. I would like my libraries and executables to only have to link to the things they directly need, not try to anticipate what the libraries they are using need.
I have also had problems when libA.so links to a static library, and when I try to compile the executable I get unresolved symbols from the static library that libA.so was supposed to be using.
I have seen a number of other people ask this and similar questions and get a variety of answers (Linking with dynamic library with dependencies), but the answers are all rather vague, often conflicting, and very much along the lines of "keep on trucking and RTFM".
I get the impression that link order matters. How so, and how do I know what order to link in?
Update
I believe that what is happening is something along the lines of libA.so contains two functions (AA and AB). libB.so needs AA and libC.so needs AB. When libB.so gets linked, g++ gets libA.so, sees that only AA is used, and drops AB. Then when libC.so is linked in, g++ sees that libA.so was already linked and doesn't revisit it, resulting in AB being undefined. I have seen documentation indicating that static libraries work this way, but would the compiler treat dynamic libraries the same way? If so, is there a way to work around it?
(You haven't shown the actual linker error, or provided nearly enough information about the problem, so what follows is partly guesswork...)
If I try to link application D just to libC.so, I get unresolved symbols for the symbols in A and B.
When linking an executable the GNU linker checks that all symbols are available. You can turn that off with --allow-shlib-undefined (to tell GCC to pass that to the linker use -Wl,--allow-shlib-undefined)
It is better not to use that option, but in that case the linker needs to know where to find libA.so and libB.so so it can check that the symbols needed by libC.so will be found. You can do that with the -rpath-link linker option
When using ELF or SunOS, one shared library may require another. This happens when an "ld -shared" link includes a shared library as one of the input files.
When the linker encounters such a dependency when doing a non-shared, non-relocatable link, it will automatically try to locate the required shared library and include it in the link, if it is not included explicitly.
So you should be able to fix the problem by using -Wl,-rpath-link,. to tell the linker to look in the current directory (.) for the libraries libC.so depends on.
I get the impression that link order matters. How so, and how do I know what order to link in?
Yes, link order matters. You should link in the obvious order ;-) If a file foo.cc depends on a library then put the library later in the linker line, so it will be found after processing foo.cc, and if that library depends on another library put that even later, so it will be processed after the earlier library that needs it. If you put a library at the start of the link line then the linker doesn't have any unresolved symbols to look up, so doesn't need to link to that library.
You need to explicitly specify all libraries that you directly use.
During static linking, the dependencies of the loaded .so are not used; when linking the main program, all symbols have to be found in either the main program itself, in a static library specified on the command line, or in a shared library specified on the command line.
This is where you get an error.
When the program is executed, the dependencies of dynamic libraries are loaded so that references from within other shared libraries can be resolved.
By the time the program runs, it might actually be linked (dynamically) against a different version of the shared library. This different version might have different dependencies, so the main program MUST NOT rely on the set of additional libraries that get loaded as dependencies.
This is why the static linker stops you early.
After some research and tests, I've found that, when linking to a library, my project needs to have the same linking option for the runtime library (MT, MD, etc) as the library I'm linking to.
What I'm wondering is, if I use a static library, (which are usually statically linked to the runtime library), am I not linking twice with the runtime library, since it's statically linked in my library and in my application? Or the linker prevents this?
Usually the static library will not be linked to the runtime library. Instead, all references to the runtime will be left unresolved, i.e. your static library file will simply contain the object files of your code, but not the object files of the runtime library.
Only when you build an actual executable using that library do you link to the runtime library, which will resolve the open references from the static library.
This is usually the default behavior when compiling static linked libraries.
That being said, with most compilers you can force the linker to resolve external references already for the static library. This may be beneficial if your library has dependencies that you don't want to pass on to compilation of the final executable.
However, if that leads to duplicate symbols, as is likely when forcing early linking to the runtime, that may break the linker. If you're lucky you'll just get a warning about duplicate symbols, but it may just as well not link at all, depending on the implementation of your linker.
How does linker know which symbols should be resolved at runtime? Particularly I'm interested what information shared object files carry that instruct linker to resolve symbols at runtime. How does the dynamic symbol resolution work at runtime, i.e. what executable will do to find the symbol and in case multiple symbols with the same name were defined which would be found?
What happens if the file was linked only statically, but then it's linked dynamically at run-time as part of a shared library? Which symbol will be used by the executable? In other words, is that possible to override symbols in an executable by linking those symbols into a shared library?
The platform in question is SUN OS.
Try the below link. I hope it answers your question
http://www.linuxjournal.com/article/6463
Check out this article from Linux Journal. For more information -- perhaps specifically related to Windows, AIX, OSx, etc -- I would recommend the Wikipedia article on Linker (computing) and the references therein.
If a file is statically linked there is no run time resolution to speak of. If a shared object links to that same library either dynamically or statically, the version linked to the library will only effect code executed in the library. This can cause problems if you link to two different versions of the same library that are incompatible and shift data back and forth.
I have a C++ executable and I'm dynamically linking against several libraries (Boost, Xerces-c and custom libs).
I understand why I would require the .lib/.a files if I choose to statically link against these libraries (relevant SO question here). However, why do I need to provide the corresponding .lib/.so library files when linking my executable if I'm dynamically linking against these external libraries?
The compiler isn't aware of dynamic linking, it just knows that a function exists via its prototype. The linker needs the lib files to resolve the symbol. The lib for a DLL contains additional information like what DLL the functions live in and how they are exported (by name, by ordinal, etc.) The lib files for DLL's contain much less information than lib files that contain the full object code - libcmmt.lib on my system is 19.2 MB, but msvcrt.lib is "only" 2.6 MB.
Note that this compile/link model is nearly 40 years old at this point, and predates dynamic linking on most platforms. If it were designed today, dynamic linking would be a first class citizen (for instance, in .NET, each assembly has rich metadata describing exactly what it exports, so you don't need separate headers and libs.)
Raymond Chen wrote a couple blog entries about this specific to Windows. Start with The classical model for linking and then follow-up with Why do we have import libraries anyway?.
To summarize, history has defined the compiler as the component that knows about detailed type information, whereas the linker only knows about symbol names. So the linker ends up creating the .DLL without type information, and therefore programs that want to link with it need some sort of metadata to tell it about how the functions are exported and what parameter types they take and return.
The reason .DLLs don't have all the information you need to link with them directly is is historic, and not a technical limitation.
For one thing, the linker inserts the versions of the libraries that exist at link time so that you have some chance of your program working if library versions are updated. Multiple versions of shared libraries can exist on a system.
The linker has the job of validating that all your undefined symbols are accounted for, either with static content or dynamic content.
By default, then, it insists on all your symbols being present.
However, that's just the default. See -z, and --allow-shlib-undefined, and friends.
Perhaps this dynamic linking is done via import libraries (function has __declspec(dllimport) before definition).
If this is the way than compilator expects that there's __imp_symbol function declared and this function is responsible for forwarding call to the right library dynamically loaded.
Those functions are generated during linkage of symbols with __declspec(dllimport) keyword
Here is a very SIMPLIFIED description that may help. Static linking puts all of the code needed to run your program into the executable so everything is found. Dynamic linking means some of the required code does not get put into the executable and will be found at runtime. Where do I find it? Is function x() there? How do I make a call to function x()? That is what the library tells the linker when you are dynamically linking.