I have a shared library that is linked with another (third-party) shared library. My shared library is then loaded using dlopen in my application. All this works fine (assuming files are in the proper path etc).
Now, the problem is that I don't even need to specify to link against the third-party shared library when I link my library. GCC accept it without reporting errors about undefined references. So, the question; how can I force GCC to notify me about undefined references?
If I change my library to be (temporarily) an executable, I get undefined references (when not supplying the library to the linker). (Works fine if I specify it.)
I.e., the following is done:
g++ -fPIC -shared -o libb.so b.o
g++ -fPIC -shared -o liba.so a.o
g++ -o a.exe a.cpp
Where the second line does NOT give out an error and the third line complains about an undefined reference.
Sample code:
a.h:
class a
{
public:
void foobar();
};
a.cpp:
#include "a.h"
#include "b.h"
void a::foobar()
{
b myB;
myB.foobar();
}
int main()
{
a myA; myA.foobar();
}
b.h:
class b
{
public:
void foobar();
};
b.cpp:
#include "b.h"
void b::foobar()
{
}
-Wl,--no-undefined linker option can be used when building shared library, undefined symbols will be shown as linker errors.
g++ -shared -Wl,-soname,libmylib.so.5 -Wl,--no-undefined \
-o libmylib.so.1.1 mylib.o -lthirdpartylib
After more research, I realized what way the stuff works. There are two linker options to manipulate undefined symbols of shared libraries:
First one is --no-undefined. It reports the unresolved symbols that are not resolved immediately, at linking stage. Unless the symbol is found in a shared library linked against, either manually (with -l switch) or automatically (libgcc_s, C++ runtime; libc, C runtime; ld-linux-**.so, dynamic linker utils) picked, --no-undefined reports it as error. That's the key the questioner needed.
There is another key, --no-allow-shlib-undefined (whose description also suggests --no-undefined). It checks if definitions in the shared libraries which you link your shared library against are satisfied. This key is of little use in the case shown in this topic, but it can be useful. However, It has its own obstacles.
The manpage provides some rationale about why it's not default:
--allow-shlib-undefined
--no-allow-shlib-undefined
Allows (the default) or disallows undefined symbols in shared
libraries (It is meant, in shared libraries _linked_against_, not the
one we're creating!--Pavel Shved). This switch is similar to --no-un-
defined except that it determines the behaviour when the undefined
symbols are in a shared library rather than a regular object file. It
does not affect how undefined symbols in regular object files are
handled.
The reason that --allow-shlib-undefined is the default is that the
shared library being specified at link time may not be the same as
the one that is available at load time, so the symbols might actually
be resolvable at load time. Plus there are some systems, (eg BeOS)
where undefined symbols in shared libraries is normal. (The kernel
patches them at load time to select which function is most appropri-
ate for the current architecture. This is used for example to dynam-
ically select an appropriate memset function). Apparently it is also
normal for HPPA shared libraries to have undefined symbols.
The thing is that what is said above is also true, for example, for Linux systems, where some of the internal routines of the shared library is implemented in ld-linux.so, the dynamic loader (it's both executable and shared library). Unless you somehow link it, you will get something like this:
/lib64/libc.so.6: undefined reference to `_dl_argv#GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `_rtld_global_ro#GLIBC_PRIVATE'
/usr/lib64/gcc/x86_64-suse-linux/4.3/libstdc++.so: undefined reference to `__tls_get_addr#GLIBC_2.3'
/lib64/libc.so.6: undefined reference to `_rtld_global#GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `__libc_enable_secure#GLIBC_PRIVATE'
These are undefined references from the loader, ld-linux.so. It is platform-specific (for example, on my system the correct loader is /lib64/ld-linux-x86-64.so). You may link the loader with your library and check even the tricky references shown above:
g++ -fPIC -shared -o liba.so a.o -Wl,--no-allow-shlib-undefined /lib64/ld-linux-x86-64.so.2
Related
I have a shared library that is linked with another (third-party) shared library. My shared library is then loaded using dlopen in my application. All this works fine (assuming files are in the proper path etc).
Now, the problem is that I don't even need to specify to link against the third-party shared library when I link my library. GCC accept it without reporting errors about undefined references. So, the question; how can I force GCC to notify me about undefined references?
If I change my library to be (temporarily) an executable, I get undefined references (when not supplying the library to the linker). (Works fine if I specify it.)
I.e., the following is done:
g++ -fPIC -shared -o libb.so b.o
g++ -fPIC -shared -o liba.so a.o
g++ -o a.exe a.cpp
Where the second line does NOT give out an error and the third line complains about an undefined reference.
Sample code:
a.h:
class a
{
public:
void foobar();
};
a.cpp:
#include "a.h"
#include "b.h"
void a::foobar()
{
b myB;
myB.foobar();
}
int main()
{
a myA; myA.foobar();
}
b.h:
class b
{
public:
void foobar();
};
b.cpp:
#include "b.h"
void b::foobar()
{
}
-Wl,--no-undefined linker option can be used when building shared library, undefined symbols will be shown as linker errors.
g++ -shared -Wl,-soname,libmylib.so.5 -Wl,--no-undefined \
-o libmylib.so.1.1 mylib.o -lthirdpartylib
After more research, I realized what way the stuff works. There are two linker options to manipulate undefined symbols of shared libraries:
First one is --no-undefined. It reports the unresolved symbols that are not resolved immediately, at linking stage. Unless the symbol is found in a shared library linked against, either manually (with -l switch) or automatically (libgcc_s, C++ runtime; libc, C runtime; ld-linux-**.so, dynamic linker utils) picked, --no-undefined reports it as error. That's the key the questioner needed.
There is another key, --no-allow-shlib-undefined (whose description also suggests --no-undefined). It checks if definitions in the shared libraries which you link your shared library against are satisfied. This key is of little use in the case shown in this topic, but it can be useful. However, It has its own obstacles.
The manpage provides some rationale about why it's not default:
--allow-shlib-undefined
--no-allow-shlib-undefined
Allows (the default) or disallows undefined symbols in shared
libraries (It is meant, in shared libraries _linked_against_, not the
one we're creating!--Pavel Shved). This switch is similar to --no-un-
defined except that it determines the behaviour when the undefined
symbols are in a shared library rather than a regular object file. It
does not affect how undefined symbols in regular object files are
handled.
The reason that --allow-shlib-undefined is the default is that the
shared library being specified at link time may not be the same as
the one that is available at load time, so the symbols might actually
be resolvable at load time. Plus there are some systems, (eg BeOS)
where undefined symbols in shared libraries is normal. (The kernel
patches them at load time to select which function is most appropri-
ate for the current architecture. This is used for example to dynam-
ically select an appropriate memset function). Apparently it is also
normal for HPPA shared libraries to have undefined symbols.
The thing is that what is said above is also true, for example, for Linux systems, where some of the internal routines of the shared library is implemented in ld-linux.so, the dynamic loader (it's both executable and shared library). Unless you somehow link it, you will get something like this:
/lib64/libc.so.6: undefined reference to `_dl_argv#GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `_rtld_global_ro#GLIBC_PRIVATE'
/usr/lib64/gcc/x86_64-suse-linux/4.3/libstdc++.so: undefined reference to `__tls_get_addr#GLIBC_2.3'
/lib64/libc.so.6: undefined reference to `_rtld_global#GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `__libc_enable_secure#GLIBC_PRIVATE'
These are undefined references from the loader, ld-linux.so. It is platform-specific (for example, on my system the correct loader is /lib64/ld-linux-x86-64.so). You may link the loader with your library and check even the tricky references shown above:
g++ -fPIC -shared -o liba.so a.o -Wl,--no-allow-shlib-undefined /lib64/ld-linux-x86-64.so.2
While linking with static library, why the order is enforced ?
g++ -ldynamic -lstatic src.cxx //ERROR
g++ -lstatic src.cxx -ldynamic //ERROR
g++ src.cxx -ldynamic -lstatic //SUCCESS
g++ -ldynamic src.cxx -lstatic //SUCCESS
Is there a technical reason why static library cannot be linked like dynamic libraries (at any order ) ?
Why linking libraries cannot be made generic (may be by mentioning while compiling/linking e.g. for static : -ls and for dynamic : -ld etc.) ?
The As Needed schism in Linux linkage
Your example:
g++ -ldynamic -lstatic src.cxx # ERROR
g++ -ldynamic src.cxx -lstatic # SUCCESS
indicates that your linux distro belongs to the RedHat clan. Let's just confirm
that, say on CentOS 7:
$ cat /proc/version
Linux version 3.10.0-693.el7.x86_64 (builder#kbuilder.dev.centos.org) \
(gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) \
#1 SMP Tue Aug 22 21:09:27 UTC 2017
$ cat foo.c
#include <stdio.h>
void foo(void)
{
puts(__func__);
}
$ cat bar.c
#include <stdio.h>
void bar(void)
{
puts(__func__);
}
$ cat main.c
extern void foo(void);
extern void bar(void);
int main(void)
{
foo();
bar();
return 0;
}
$ gcc -Wall -fPIC -c foo.c
$ gcc -shared -o libfoo.so foo.o
$ gcc -Wall -c bar.c
$ ar cr libbar.a bar.o
$ gcc -Wall -c main.c
$ gcc -o prog -L. -lfoo -lbar main.o -Wl,-rpath=$(pwd)
main.o: In function `main':
main.c:(.text+0xa): undefined reference to `bar'
collect2: error: ld returned 1 exit status
# :(
$ gcc -o prog -L. -lfoo main.o -lbar -Wl,-rpath=$(pwd)
$ # :)
$ ./prog
foo
bar
So you're right there.
Now let's check it out on a distro from the Debian clan:
$ cat /proc/version
Linux version 4.13.0-32-generic (buildd#lgw01-amd64-016) \
(gcc version 7.2.0 (Ubuntu 7.2.0-8ubuntu3)) \
#35-Ubuntu SMP Thu Jan 25 09:13:46 UTC 2018
Here, it all goes the same as far as:
$ gcc -o prog -L. -lfoo -lbar main.o -Wl,-rpath=$(pwd)
main.o: In function `main':
main.c:(.text+0x5): undefined reference to `foo'
main.c:(.text+0xa): undefined reference to `bar'
collect2: error: ld returned 1 exit status
when it gets different. Now the linkage can't resolve either foo - from the
shared library libfoo.so - or bar - from the static library libbar.a. And
to fix that we need:
$ gcc -o prog -L. main.o -lfoo -lbar -Wl,-rpath=$(pwd)
$ ./prog
foo
bar
with all the libraries mentioned after the object file(s) - main.o - that
reference the symbols they define.
The Centos-7 (RedHat) linkage behaviour is old-school. The Ubuntu 17.10 (Debian)
linkage behaviour was introduced in Debian 7, 2013, and trickled down
to the Debian-derived distros. As you see, it abolishes the distinction
between shared libraries and static libraries as regards the library needing,
or not needing, to appear in the linkage sequence after all input
files that reference it. They all must appear in dependency order (DO1),
shared libraries and static libraries alike.
This comes down to how the distro decides to build their version of the GCC
toolchain - how they choose the default options that get passed to the system
linker (ld) behind the scenes when it is invoked by one of the language
front-ends (gcc, g++, gfortran etc.) to execute a linkage for you.
Specifically, it comes down to whether the linker option --as-needed
is, or is not, inserted by default into the ld commandline before the libraries
are inserted.
If --as-needed is not in effect then a shared library libfoo.so is arrived at,
then it will be linked regardless of whether the linkage has so far accrued any
unresolved references to symbols that the shared library defines. In short,
it will be linked regardless of any proven need to link it. Maybe the further
progess of the linkage, to subsequent inputs, will give rise to unresolved references
that libfoo.so resolves, justifying its linkage. But maybe not. It gets linked
anyhow. That's the RedHat way.
If --as-needed is in effect when a libfoo.so is arrived at, then it
will be linked if and only if it exports a definition for at least one symbol
to which an unresolved reference has already accrued in the linkage, i.e.
there is a proven need to link it. It cannot end up linked if there is
no need to link it. That's the Debian way.
The RedHat way with shared library linkage was prevalent until Debian 7
broke ranks. But the linkage of static libraries has always conformed to the as needed principle
by default. There's no --as-needed option that applies to static libraries.
Instead there's the opposite, --whole-archive:
you need to use that to override the default behaviour and link object files from static libraries regardless of need.
So folks like you, in RedHat land, observe this puzzling difference: by default static libaries
have to be linked in DO; for shared libraries, any order will do by default.
Folks is Debian land see so such difference.
Wny so?
Since the Redhat way has this puzzling difference - a stumbling block for
the linkage efforts of the uninitiated - it's natural to ask why, historically,
it was as needed for static libraries, but not as needed for shared libraries,
as a matter of course, and why it still goes in RedHat land.
Simplifying grossly, the linker assembles a program (or shared library) by
incrementally populating sections and dynamic dependency records (DDRs2) in
a structure of sections and DDRs that starts off empty and
ends up being a binary file that the OS loader can parse and successfully map
into a process address space: e.g. an ELF executable or DSO. (Section
here is a genuine technical term. Dynamic dependency record isn't.
I've just coined now for convenience.)
Loosely speaking, the linker inputs that drive this process are object files,
shared libraries, or static libraries. But strictly speaking, they are either
object files or shared libraries. Because a static libary is simply
an ar archive of files that happen to be
object files. As far as the linker is concerned it is just a sequence of object
files that it might or might not need to use, archived together with a symbol table
by which the linker can query which of the object files, if any, defines a symbol.
When the linker arrives at an object file, that object file is always linked
into the program. The linker never asks whether it needs an object file
(whatever that might mean). Any object file is an unconditional source of linkage
needs, that further inputs have to satisfy.
When an object file is input, the linker has to dismantle it into
the input sections of which it composed and merge them into the output
sections in the program. When an input section S appears in one object
file, the chances are that a section S will appear in other object files;
maybe all of them. The linker has to stitch together all the input S sections
into a single output S section in the program, so it isn't finally done
with composing an output section S till the linkage is finished.
When a shared library libfoo.so is input to the linkage, the linker outputs
a DDR into the program (if it decides that the library is needed, or doesn't care). That is essentialy a memo that will be read at runtime by
the loader, telling it that libfoo.so is a dependency of the process that is
under construction; so it will locate libfoo.so by its standard search algorithm,
load it and map it into the process.
Consuming an object file is a relatively costly bit of linkage; consuming
a shared library is relatively cheap - especially if the linker does not
have to bother beforehand figuring out whether the shared library is needed.
The input/output section processing of an object file is typically more bother than writing out a DDR.
But more important than the effort, linking an object file typically makes the program signficantly
larger, and can make it arbitarily larger. Linking a shared library adds
only a DDR, which is always a tiny thing.
So there's a respectable rationale for a linkage strategy to reject the linking of
an object file unless it is needed, but to tolerate the linking of a shared library
without need. Linking an unnecessary object file adds an arbitrary amount of dead
weight to the program, at a proportional burden on the linkage. But
if the linker doesn't have to prove that a shared library is needed, then it
can link it in a jiffy adding negligibly to the bulk of the program. And if the developer has chosen to add the shared library to the linkage, chances are good it will be needed. The RedHat
camp still thinks that rationale is good enough.
The Debian camp also has a respectable rationale of course. Yes, a Debian linkage
involves the extra effort of determining whether libfoo.so, when it is
reached, defines any symbol to which there is an unresolved reference at that
point. But by only linking shared libraries that are needed: -
At runtime, the loader is spared the wastage of either loading redundant
dependencies, or figuring out that they are redundant so as not to load them.
Package management with respect to runtime dependencies is eased if
redundant runtime dependencies are weeded out at linktime.
Developers, like you, do not get tripped up by the inconsistent linkage rules
for static and shared libraries! - a snag that's aggravated by the fact that
-lfoo in the linker commandline does not reveal whether it will resolve
to libfoo.so or libfoo.a.
There are thornier pros and cons for each side in the schism.
Now consider how the linker uses a static library, libabc.a - a list of object files a.o, b.o, c.o.
The as needed principle is applied, like this: When the linker arrives at libabc.a,
it has 0 or more unresolved symbol references in hand that it has carried
forward from the 0 more object files and shared libraries it has already linked
into the program. The linker's question is: Are there any object files in
this archive that provide definitions for any of these unresolved symbol references?
If there are 0 such unresolved references, then the answer is trivially No. So
there's no need to look in the archive. libabc.a is passed over. The linker moves
on to the next input. If it has got some unresolved symbol references in hand, then the
linker inspects the symbols that are defined by the object files in the archive. It
extracts just those object files - if any - that provide symbol definitions that it needs
3 and inputs those object files to the linkage, exactly as if they were individually
named in the commandline and libabc.a was not mentioned at all. Then it moves
it on to the next input, if any.
It's obvious how the as needed principle for static libraries implies DO. No
object file will be extracted from a static library and linked unless an unresolved
reference to some symbol that the object file defines has accrued from some
object file (or shared library) already linked.
Must static libraries be As Needed ?
In RedHat land, where DO is waived for shared libraries, what we do in
its absence is just link every shared library that is mentioned. And as we've
seen, this is tolerably cheap in linkage resource and program size. If we
also waived DO for static libraries, the equivalent strategy would
be to link every object file in every static library that is mentioned. But
that is exorbitantly costly, in linkage resource and program dead weight.
If we wanted to be free of DO for static libraries, but still not link
object files with no need, how could the linker proceed?
Maybe like this?:-
Link all of the object files that are explicitly mentioned into the program.
Link all the shared libraries mentioned.
See if there remain any unresolved references. If so, then -
Extract all of the object files from all of the static libraries that are mentioned
into a pool of optional object files.
Then carry on the linkage on an as needed basis against this pool of optional
object files, to success or failure.
But nothing like this will fly. The first definition of a symbol that the linker
sees is the one that's linked. This means that the order in which object files
are linked matters, even as between two different linkage orders that are both
successful.
Say that object files a.o, b.o have already been linked; unresolved references
remain and then the linker has a choice of optional object files c.o, d.o, e.o, f.o
to continue with.
There may be more than one ordering of c.o, d.o, e.o, f.o that
resolves all references and gives us a program. It might be the case that linking,
say, e.o first resolves all outstanding references and yields no new ones,
giving a program; while linking say c.o first also resolves all outstanding
references but produces some new ones, which require the linking of some or
all of d.o, e.o, f.o - depending on the order - with each possible linkage
resulting in yet another different program.
That's not all. There may be more that one ordering of c.o, d.o, e.o, f.o such that, after some object file is linked - point P - all
previously outstanding references are resolved, but where:-
Some of those orderings either produce no new references at point P or produce only references that some further linkage order can resolve.
Other ones produce new references at point P that no further linkage order can resolve.
So, whenever the linker discovers it has made a type 2 choice at some earlier point, it would need
to backtrack to that point and try one of the other choices that were then available, that it hasn't already tried,
and only conclude that the linkage fails when it has tried them all unsuccessfully.
Linking like this against a pool of N optional object files will take time proportional
to factorial N to fail.
Living with DO for static libraries as we do now, we specify object files and/or static libraries Ij in the
linker commandline in some order:
I0, I1, ... In
and this equates to an ordering of object files which for argument's sake might
ressemble:
O0, O1, [02,... O2+j], O2+j+1, [O2+j+2,... O2+j+k] ...
where [Oi...] is a sub-sequence of optional object files (i.e. a static library) that will be
available to the linker at that point.
Whether we know it not when we compose the commandline, we are asserting not just that this order is
a good DO ordering that can be linked to yield some program, but also that this
ordering yields the program that we intend.
We might be mistaken on the first count ( = linkage failure). We might even be
mistaken on the second ( = a mean linkage bug). But if we stop caring about the order of these
inputs and just leave it to the linker somehow to find a good DO over them, or prove that there isn't one,
then:
We have actually stopped caring about which program we will get, if any.
We have stopped caring about whether linkage will terminate in any feasible time.
This is not going to happen.
Couldn't we get a warning for broken DO?
In a comment you asked why the linker could not at least warn us if our
static object files and static libraries are not in DO.
That would be in addition to failing the linkage, as it does now. But to give us this
additional warning the linker would have to prove that the linkage failed
because the object files and static libraries are not in DO, and not just because
there are references in the linkage that nothing in the linkage defines. And it
could only prove that the linkage failed because of broken DO by proving that
some program can be linked by some permutation of the object files and static libraries.
That's a factorial-scale task, and we don't care if some program can be linked,
if the program we intend to link can't be: the linker has no information about
what program we intend to link except the inputs we give it, in the order that
we give them.
It would be easy make the linker (or more plausibly, the GCC frontends) emit a warning if any library was mentioned
before any object file in the commandline. But it would have some nuisance value, because
such a linkage isn't necessarily going to fail and might in fact be the intended
linkage. "Libraries after object files" is just pretty good guidance for routine
invocations of the linker via a GCC frontend. Worse, such a warning would only be practical for object files after libraries and not for cases of broken DO between libraries, so it would only do some of the job.
[1] My abbreviation.
[2] Also my abbreviation.
[3] More precisely, object file extraction from a static library is recursive.
The linker extracts any object files that define unresolved references it
already had in hand, or any new unresolved references that accrue while
linking object files extracted from the library.
When the linker loads the library static, it will see if any symbols from it are needed. It will use the symbols that are needed, and discard the rest. That of course means that if no symbols are needed from the library then all are discarded.
This is the reason that putting the library in front of the object files that depends on it will not work.
As a rule of thumb, always place libraries (even dynamic) at the end of the command line. And in order of dependencies. If module (object file or library) A depend on module B, always put A before B.
I have two simple files:
runner.cpp:
#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MODULE Main
#include <boost/test/unit_test.hpp>
and test1.cpp:
#define BOOST_TEST_DYN_LINK
#ifdef STAND_ALONE
# define BOOST_TEST_MODULE Main
#endif
#include <boost/test/unit_test.hpp>
BOOST_AUTO_TEST_SUITE( Foo)
BOOST_AUTO_TEST_CASE( TestSomething )
{
BOOST_CHECK( true );
}
BOOST_AUTO_TEST_SUITE_END()
To compile, I'm using:
$ g++ -I/e/code/boost_1_52_0 -o runner -lboost_unit_test_framework runner.cpp test1.cpp
I get the following error:
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\ccU0cDSz.o:runner.cpp:(.text+0x8c): multiple definition of `main'
c:/pdev/mingw/bin/../lib/gcc/i686-pc-mingw32/4.7.2/../../../libboost_unit_test_framework.a(unit_test_main.o):unit_test_main.cpp:(.text.startup+0x0): first defined here
c:/pdev/mingw/bin/../lib/gcc/i686-pc-mingw32/4.7.2/../../../libboost_unit_test_framework.a(unit_test_main.o):unit_test_main.cpp:(.text.startup+0x14): undefined reference to `init_unit_test_suite(int, char**)'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\ccU0cDSz.o:runner.cpp:(.text+0x52): undefined reference to `_imp___ZN5boost9unit_test9framework17master_test_suiteEv'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\ccU0cDSz.o:runner.cpp:(.text+0xb0): undefined reference to `_imp___ZN5boost9unit_test14unit_test_mainEPFbvEiPPc'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\ccU0cDSz.o:runner.cpp:(.text$_ZN5boost9unit_test13test_observerD2Ev[__ZN5boost9unit_test13test_observerD2Ev]+0xe): undefined reference to `_imp___ZTVN5boost9unit_test13test_observerE'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\ccU0cDSz.o:runner.cpp:(.text$_ZN5boost9unit_test13test_observerC2Ev[__ZN5boost9unit_test13test_observerC2Ev]+0xe): undefined reference to `_imp___ZTVN5boost9unit_test13test_observerE'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\ccU0cDSz.o:runner.cpp:(.text$_ZN5boost9unit_test15unit_test_log_tC1Ev[__ZN5boost9unit_test15unit_test_log_tC1Ev]+0x22): undefined reference to `_imp___ZTVN5boost9unit_test15unit_test_log_tE'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\cciSdkmB.o:test1.cpp:(.text+0x88): undefined reference to `_imp___ZN5boost9unit_test15unit_test_log_t14set_checkpointENS0_13basic_cstringIKcEEjS4_'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\cciSdkmB.o:test1.cpp:(.text+0x136): undefined reference to `_imp___ZN5boost10test_tools9tt_detail10check_implERKNS0_16predicate_resultERKNS_9unit_test12lazy_ostreamENS5_13basic_cstringIKcEEjNS1_10tool_levelENS1_10check_typeEjz'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\cciSdkmB.o:test1.cpp:(.text+0x21d): undefined reference to `_imp___ZN5boost9unit_test9ut_detail24auto_test_unit_registrarC1ENS0_13basic_cstringIKcEE'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\cciSdkmB.o:test1.cpp:(.text+0x284): undefined reference to `_imp___ZN5boost9unit_test9ut_detail24auto_test_unit_registrarC1EPNS0_9test_caseEm'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\cciSdkmB.o:test1.cpp:(.text+0x2a4): undefined reference to `_imp___ZN5boost9unit_test9ut_detail24auto_test_unit_registrarC1Ei'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\cciSdkmB.o:test1.cpp:(.text$_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[__ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x1d): undefined reference to `_imp___ZN5boost9unit_test9ut_detail24normalize_test_case_nameENS0_13basic_cstringIKcEE'
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\cciSdkmB.o:test1.cpp:(.text$_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[__ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x5b): undefined reference to `_imp___ZN5boost9unit_test9test_caseC1ENS0_13basic_cstringIKcEERKNS0_9callback0INS0_9ut_detail6unusedEEE'
collect2.exe: error: ld returned 1 exit status
I'm using g++ 4.7.2 on MinGW, with boost 1.52.0.
I get the same errors when only trying to compile test1.cpp - except the "multiple main definition" one.
I perused the official documentation for quite a while, but its scarce on details regarding linking options. When I compiled the boost libs, besides unit_test_framework, I also got prg_exec_monitor and test_exec_monitor; perhaps I should link these somehow ? I tried many combinations, but all resulted in some kind of undefined reference linker error.
Complete list of boost generated libraries - I have them all in the project root:
libboost_prg_exec_monitor-mgw47-mt-1_52.a
libboost_prg_exec_monitor-mgw47-mt-1_52.dll
libboost_prg_exec_monitor-mgw47-mt-1_52.dll.a
libboost_prg_exec_monitor-mgw47-mt-d-1_52.a
libboost_prg_exec_monitor-mgw47-mt-d-1_52.dll
libboost_prg_exec_monitor-mgw47-mt-d-1_52.dll.a
libboost_test_exec_monitor-mgw47-mt-1_52.a
libboost_test_exec_monitor-mgw47-mt-d-1_52.a
libboost_unit_test_framework-mgw47-mt-1_52.a
libboost_unit_test_framework-mgw47-mt-1_52.dll
libboost_unit_test_framework-mgw47-mt-1_52.dll.a
libboost_unit_test_framework-mgw47-mt-d-1_52.a
libboost_unit_test_framework-mgw47-mt-d-1_52.dll
libboost_unit_test_framework-mgw47-mt-d-1_52.dll.a
With help from #llonesmiz, a number of issues were identified.
1. Libraries need to be specified after objects and sources which use them.
As described here:
The traditional behavior of linkers is to search for external functions from
left to right in the libraries specified on the command line. This means that a
library containing the definition of a function should appear after any source
files or object files which use it. This includes libraries specified with the
short-cut -l option, as shown in the following command:
$ gcc -Wall calc.c -lm -o calc (correct order)
With some linkers the opposite ordering (placing the -lm option before the file
which uses it) would result in an error,
$ cc -Wall -lm calc.c -o calc (incorrect order)
main.o: In function 'main':
main.o(.text+0xf): undefined reference to 'sqrt'
because there is no library or object file containing sqrt after ‘calc.c’. The
option -lm should appear after the file ‘calc.c’
2. Library paths should be explicitly specified.
If no lib paths are specified, the linker might look for the libs in a series
of default folders, thus loading a different library then intended. This is what
happened in my case - I wanted to link boost_unit_test_framework, but did not
specify a path because I assumed the linker would look in the current folder.
That's what happens at runtime, after all - if the dll is in the same folder
with the exe, it will find it.
I found it a little bit strange the linker would find the lib, since it was
named ibboost_unit_test_framework-mgw47-mt-1_52.dll. When I tried to link to
a non-existing lib, the linker complained though, so I assumed this isn't an
issue, and MinGW 's linker ignores those suffixes.
After some more research, I found this article about MinGW library paths.
The folders MinGW searches for libs can be found in the output of gcc -print-search-dirs.
The article also contains some bash magic to make sense of that output:
gcc -print-search-dirs | sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,; ,g' | tr \; \\012 | grep -v '^ */'
This will print a nice list of those folders. gcc will not, by default,
look in the current directory for libs. I looked in each of them, and found the
lib that was being loaded - libboost_unit_test_framework.a, a static lib.
This brings into light another issue worth mentioning:
3. Static versus dynamic linking
I did not specify whether I want boost_unit_test_framework linked statically or dynamically.
In this case, gcc prefers dynamic linking:
Because of these advantages gcc compiles programs to use shared libraries by
default on most systems, if they are available. Whenever a static library
‘libNAME.a’ would be used for linking with the option -lNAME the compiler
first checks for an alternative shared library with the same name and a ‘.so’
extension.
(so is the extension for dynamic libraries on Unix - on Windows, the equivalent is dll.)
So, what happened is that gcc looked for libboost_unit_test_framework.dll
in all it's default folders, but couldn't find it. Then it looked for
libboost_unit_test_framework.a, and statically linked that. This resulted in
linking errors because the sources have #define BOOST_TEST_DYN_LINK, and
therefore expect to have the lib dynamically linked.
To enforce static or dynamic linking, the -Wl,-Bstatic and -Wl,-Bdynamic
linker options come into play, described here.
If I tell the linker that I want dynamic linking:
$ g++ -I/e/code/boost_1_52_0 runner.cpp test1.cpp -o runner -Wl,Bdynamic -lboost_unit_test_framework
This will fail, because the linker will not be able to find the dll.
4.Summary
The issues were:
libraries where specified before the sources which used them
the lib path wasn't specified
the type of linking wasn't specified
the name of the library was not correct
Final, working command:
$ g++ -I/e/code/boost_1_52_0 -o runner runner.cpp test1.cpp -L. -Wl,-Bdynamic -lboost_unit_test_framework-mgw47-mt-1_52
I've been working with and testing a self-registering, abstract factory based upon the one described here:
https://stackoverflow.com/a/582456
In all my test cases, it works like a charm, and provides the features and reuse I wanted.
Linking in this factory in my project using cmake has been quite tricky (though it seems to be more of an ar problem).
I have the identical base.hpp, derivedb.hpp/cpp, and an equivalent deriveda.hpp/cpp to the example linked. In main, I simply instantiate the factory and call createInstance() twice, once each with "DerivedA" and "DerivedB".
The executable created by the line:
g++ -o testFactory main.cpp derivedb.o deriveda.o
works as expected. Moving my derived classes into a library (using cmake, but I have tested this with ar alone as well) and then linking fails:
ar cr libbase.a deriveda.o derivedb.o
g++ -o testFactory libbase.a main.cpp
only calls the first static instantiation (from derivedA.cpp) and never the second static instantiation, i.e.
// deriveda.cpp (if listed first in the "ar" line, this gets called)
DerivedRegister<DerivedA> DerivedA::reg("DerivedA");
// derivedb.cpp (if listed second in the "ar" line, this does not get called)
DerivedRegister<DerivedB> DerivedB::reg("DerivedB");
Note that swapping the two in the ar line calls only the derivedb.cpp static instantiation, and not the deriveda.cpp instantiation.
Am I missing something with ar or static libraries that somehow do not play nice with static variables in C++?
Contrary to intuition, including an archive in a link command is not the same as including all of the objects files that are in the archive. Only those object files within the archive necessary to resolve undefined symbols are included. This is a good thing if you consider that once there was no dynamic linking and otherwise the entirety of any libraries (think the C library) would be duplicated into each executable. Here's what the ld(1) manpage (GNU ld on linux) has to say:
The linker will search an archive only once, at the location where it is specified on the command line. If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an object appearing later on the command line will not cause the linker to search the archive again.
Unfortunately there's no standard way to include every member of an archive in the linked executable. On linux you can use g++ -Wl,-whole-archive and on Mac OS X you can use g++ -all_load.
So with GNU binutils ld, the link command should be
g++ -o testFactory -Wl,-whole-archive libbase.a -Wl,-no-whole-archive main.cpp
the -Wl,-no-whole-archive ensures that any archive appearing later in the final link command generated by g++ will be linked in the normal way.
Say I had a library called libfoo which contained a class, a few static variables, possibly something with 'C' linkage, and a few other functions.
Now I have a main program which looks like this:
int main() {
return 5+5;
}
When I compile and link this, I link against libfoo.
Will this have any effect? Will my executable increase in size? If so, why? Do the static variables or their addresses get copied into my executable?
Apologies if there is a similar question to this or if I'm being particularly stupid in any way.
It won't do anything in a modern linker, because it knows the executable doesn't actually use libfoo's symbols. With gcc 4.4.1 and ld 2.20 on my system:
g++ linker_test.cpp -static -liberty -lm -lz -lXp -lXpm -o linker_test_unnecessary
g++ linker_test.cpp -static -o linker_test_none
ls -l linker_test_unnecessary linker_test_none
They are both 626094 bytes. Note this also applies to dynamic linking, though the size they both are is much lower.
A library contains previously compiled object code - basically a static library is an archive of .o or .obj files.
The linker looks at your object code and sees if there are any unresolved names and if so looks for these in the library, if it finds them it includes the object file that contains them and repeats this.
Thus only the parts of the static library that are needed are included in your executable.
Thus in your case nothing from libfoo will be added to you executable