When compiling a program with static libraries, it was suggested to me from many sources (including SO community) to include the library twice.
As in:
gcc main.c -lslA -lslB -lslC -lslA -lslB -o final
Does this result in a bigger executable (.i.e. is the linker smart enough to avoid double inclusion?).
Is this (multiple inclusion) the proper solution or a workaround (.i.e. will there always exist a more proper, even if harder way to handle it)
The only reason to include the library multiple times is, for example, if slA requires a symbol resolved by slB but slB requires a symbol required by slA. The linker does a single pass to resolve symbols, but repeating your library causes, in effect, a second pass on that library. It won't change the size of your output, but it's not necessary either:
Instead of presenting your libraries multiple times, you can tell the gcc linker to group certain libraries together -- letting it do what it needs to resolve the symbols within that group. For example:
gcc main.c -Wl,--start-group -lslA -lslB -lslC -Wl,--end-group -o final
Related
I was recently reading this answer and noticed that it seems inconvenient for users to have to link static libraries in the correct order.
Is there some flag or #pragma I can pass to gcc when compiling my library so that my library's object files will always be included?
To be more specific, I want it to be case that the user can link my static library before another library that depends on it, and not wind up with unresolved symbols, in the same way that object files which are explicitly specified on the linker line are.
In particular, I am looking for solutions where the user of the library does not need to do anything special, and merely needs to pass -lMylibrary the way they would link any other library.
Is there some flag or #pragma I can pass to gcc
No.
I want it to be case that the user can link my static library before another library that depends on it, and not wind up with unresolved symbols, in the same way that object files which are explicitly specified on the linker line are.
Ship your "library" as a single object file. In other words, instead of:
ar ru libMyLibrary.a ${OBJS}
use:
ld -r -o libMyLibrary.a ${OBJS}
In particular, I am looking for solutions where the user of the library does not need to do anything special, and merely needs to pass -lMylibrary the way they would link any other library.
You can name your object file libMyLibrary.a. I believe the linker will search for it using the usual rules, but when it finds it, it will discover that this is an object file, and treat it as such, despite it being "misnamed". This should work at least on Linux and other ELF platforms. I am not sure whether it will work on Windows.
Recently, I was trying to build an application, which uses some libraries, available in form of shared object files. I wasted lot of time in compiling the CPP code and it didn't work.
Below is the command, previously I was trying to compile the code-
g++ -I/opt/ros/indigo/include/ -I/usr/include/eigen3/ -L/opt/ros/indigo/lib/ -lorocos-kdl -lkdl_parser test.cpp -o test
The above command always shows many undefined references errors. Just for the curiosity, I changed the order of parameters. Below is the command, which is working-
g++ -L/opt/ros/indigo/lib -I/opt/ros/indigo/include -I/usr/include/eigen3 test.cpp -lorocos-kdl -lkdl_parser -o test
I posted the complete code and solution here.
My question is why does the order of passing parameters to g++ matter? Is there any alternative to avoid such problems in future?
Generally the order of arguments doesn't matter, but there are of course exceptions. For example if you provide multiple -O flags it will be the last one that is used, the same for other flags.
Libraries are a little different though, because for them the order is significant. If object file or library A depends on library B, then A must come before B on the command line. This is because of how the linker scans for symbols: When you use a library the linker will check if there are any symbols that could be resolved. Once this scan is over the library is discarded and will not be searched again.
This means when you have -lorocos-kdl -lkdl_parser test.cpp the linker will scan the libraries orocos-kdl and kdl_parser first, notice that there aren't dependencies on these library, no symbols from the libraries are needed, and continue with the object file generated by the source file.
When you change the order to test.cpp -lorocos-kdl -lkdl_parser the linker will be able to resolve the undefined symbols referenced by test.cpp when it comes to the libraries.
You can (at least in some versions of gcc) use parenthesis around the libraries if you don't want to care about the order.
See:
Why does the order in which libraries are linked sometimes cause errors in GCC?
Specifically:
If a static library depends on another library, but the other library
again depends on the former library, there is a cycle. You can resolve
this by enclosing the cycling dependent libraries by -( and -), such
as -( -la -lb -) (you may need to escape the parens, such as -( and
-)). The linker then searches those enclosed lib multiple times to ensure cycling dependencies are resolved.
I asked myself the following question, when I was discussing this topic .
Are there cases when some unused code from translation units will link to final executable code (in release mode of course) for popular compilers like GCC and VC++?
For example suppose we have 2 compilation units:
//A.hpp
//Here are declarations of some classes, functions, extern variables etc.
And source file
//A.cpp
//defination of A.hpp declarations
And finally main
//main.cpp
//including A.hpp library
#include "A.hpp"
//here we will use some stuff from A.hpp library, but not everything
My question is. What if in main.cpp not all the stuff from A.hpp is used? Will the linker remove all unused code, or there are some cases, when some unused code can link with executable file?
Edit: I'm interested in G++ and VC++ linkers.
Edit: Of course I mean in release mode.
Edit: I'm starting bounty for this question to get good and full answer. I'm expecting answer, which will explain in which cases g++ and VC++ linkers are linking junk and what kind of code they are able to remove from executable file(unneeded functions, unneeded global variables, unneeded class definitions, etc...) and why aren't they able to remove some kind of unneeded stuff.
As other posters have indicated, the linker typically does not remove dead code before building the final executable. However, there are often Optimization settings you can use to force the linker to try extra hard to do this.
For GCC, this is accomplished in two stages:
First compile the data but tell the compiler to separate the code into separate sections within the translation unit. This will be done for functions, classes, and external variables by using the following two compiler flags:
-fdata-sections -ffunction-sections
Link the translation units together using the linker optimization flag (this causes the linker to discard unreferenced sections):
-Wl,--gc-sections
So if you had one file called test.cpp that had two functions declared in it, but one of them was unused, you could omit the unused one with the following command to gcc(g++):
gcc -Os -fdata-sections -ffunction-sections test.cpp -o test.o -Wl,--gc-sections
(Note that -Os is an additional linker flag that tells GCC to optimize for size)
I have also read somewhere that linking static libraries is different though. That GCC automatically omits unused symbols in this case. Perhaps another poster can confirm/disprove this.
As for MSVC, as others have mentioned, function level linking accomplishes the same thing.
I believe the compiler flag for this is (to sort things into sections):
/Gy
And then the linker flag (to discard unused sections):
/OPT:REF
EDIT: After further research, I think that bit about GCC automatically doing this for static libraries is false.
The linker will not remove code.
You can still access it via dlsym dynamically in your code.
In general, linkers tend to include everything from the object files explicitly passed on the command line, but only pull in those object files from a static library that contain symbols needed to resolve external references from object files already linked.
However, a linker may decide to discard functions that are never called, or data which is never referenced. The precise details will depend on the compiler and linker switches.
In C++ code, if a source file is explicitly compiled and linked in to your application then I would expect that the objects with static storage duration that have constructors and/or destructors will be included, and their constructors/destructors run at the appropriate times. Consequently, any code called from those constructors or destructors must be in the final executable. However, if the code is not called from anywhere then you cannot write a program to tell whether or not the code is included without using things like dlsym, so the linker may well omit to include it in the final executable.
I would also expect that any symbols defined with global visibility such that they could be found via dlsym (as opposed to "hidden" symbols which are only visible within the executable) would be present in the final executable. However, this is an expectation rather than something I have confirmed by testing or reading the docs.
If you wanted to ensure code was in your executable even if it isn't called by inside it, you could load it in as a statically aware dynamic link library (a statically aware library is one which is loaded automatically into memory as the program is loaded, as opposed to the functionality where you can pass a string to a function that loads a library and then you manually search for hooks)
It appears GCC linker doesn't care for one variable being defined in two files. I suspect this is the cause of trouble a 3rd party library is causing us.
Take this:
File a.cpp contains:
int foo;
//do things with it.
File b.cpp contains:
int foo;
//do other things with it.
File c.cpp contains:
extern int foo;
//do other things with it.
They are all compiled by gcc to .o files, then linked as shared object.
gcc -fPIC -c a.cpp
gcc -fPIC -c b.cpp
gcc -fPIC -c c.cpp
ld *.o -shared -soname,mylib -o mylib
The linker doesn't complain at all, but the resulting binary misbehaves. We suspect there are at least a few conflicts of this kind and would like to locate them. What kind of linker options would let us detect them?
(interestingly, if the variables are initialized (int foo=0) in both files, it produces an error).
Hold on now -- are you using foo for two different purposes in the two files? That would certainly lead to run-time errors. If foo needs to be global, then it should be defined in just one module -- the linker may accept it, but you will still only get one copy of foo. If it doesn't need to be global, it should be declared 'static int foo;'
This is a serious design bug in gcc/ld, it does not occur using MSVC. It won't happen linking programs, only shared libraries. When you link a program, the linker ensures all external references are satisfied, at least at link time. When you link a shared library it does not. Instead, external references for which there are no definitions are just left to dangle, the argument (given in ld man page) being that the symbol has to be resolved at load-time dynamically anyhow, so there's no point checking it. It's also hard if you use the stupid feature of shared libraries grabbing symbols from executables.
Your program will not misbehave. If you specify that symbols in a library must be satisfied on loading, you will get a load-time error, if you specify lazy linkage, the error will still occur, but only on the first use of the symbol (AFAIK!)
Some older OS, I believe BSD for example, allowed unsatisfied external pointers to be left as NULL so that you could write an "in program" check to see if the symbol was linked or not. Linux ld at least does not support this AFAIK.
There is a linker switch to force satisfaction of external references for shared libraries, but it is hard to use correctly in portable builds because it requires you to explicitly link the startup library for your processor.
I consider this a very serious design bug, and tried to file a bug report. In my own product we were happy to be able to build under Cygwin because underneath it uses MSVC linker which does not permit this behaviour, quite a lot of bugs were found in my code this way.
It seems compiler option -fno-common forces all the variables to be initialized, so it triggers errors upon linking.
Prior to today I had always believed that the order that objects and libraries were passed to g++ during the linking stage was unimportant. Then, today, I tried to link from c++ code to c code. I wrapped all the C headers in an extern "C" block but the linker still had difficulties finding symbols which I knew were in the C object archives.
Perplexed, I created a relatively simple example to isolate the linking error but much to my surprise, the simpler example linked without any problems.
After a little trial and error, I found that by emulating the linking pattern used in the simple example, I could get the main code to link OK. The pattern was object code first, object archives second eg:
g++ -o serverCpp serverCpp.o algoC.o libcrypto.a
Can anyone shed some light on why this might be so?. I've never seen this problem when linking ordinary c++ code.
The order you specify object files and libraries is VERY important in GCC - if you haven't been bitten by this before you have lead a charmed life. The linker searches symbols in the order that they appear, so if you have a source file that contains a call to a library function, you need to put it before the library, or the linker won't know that it has to resolve it. Complex use of libraries can mean that you have to specify the library more than once, which is a royal pain to get right.
The library order pass to gcc/g++ does actually matter. If A depends on B, A must be listed first. The reason is that it optimizes out symbols that aren't referenced, so if it sees library B first, and no one has referenced it at that point then it won't link in anything from it at all.
A static library is a collection of object files grouped into an archive. When linking against it, the linker only chooses the objects it needs to resolve any currently undefined symbols. Since the objects are linked in order given on the command line, objects from the library will only be included if the library comes after all the objects that depend on it.
So the link order is very important; if you're going to use static libraries, then you need to be careful to keep track of dependencies, and don't introduce cyclic dependencies between libraries.
You can use --start-group archives --end-group
and write the 2 dependent libraries instead of archives:
gcc main.o -L. -Wl,--start-group -lobj_A -lobj_b -Wl,--end-group