I wanted to introduce a weak symbol into my code, however, I am unable to comprehend its behavior when *.a files are used.
This is my minimal example:
file a.h:
void foo() __attribute__((weak));
file a.c:
#include "a.h"
#include <stdio.h>
void foo() { printf("%s\n", __FILE__); }
file b.c:
#include <stdio.h>
void foo() { printf("%s\n", __FILE__); }
file main.cpp:
#include "a.h"
#include <stdio.h>
int main() { if (foo) foo(); else printf("no foo\n"); }
Now, depending if I use *.o files (make -c a.c and make -c b.c) or *.a files (ar cr a.o and ar cr b.o) the output is different:
1) g++ main.cpp a.o b.o prints b.c
2) g++ main.cpp b.o a.o prints b.c
3) g++ main.cpp a.a b.a prints no foo
4) g++ main.cpp b.a a.a prints no foo
1), 2) work just fine but the output for 3), 4) seems to be a little unexpected.
I was desperately trying to make this example work with archives so I made few changes:
file a.h:
void foo();
file a.c:
#include "a.h"
#include <stdio.h>
void __attribute__((weak)) foo() { printf("%s\n", __FILE__); }
After this modification:
1) g++ main.cpp a.a b.a prints a.c
2) g++ main.cpp b.a a.a prints b.c
So it works a bit better. After running nm a.a shows W _Z3foov so there is no violation of ODR. However, I don't know if this is a correct usage of weak attribute. According to gcc documentation:
The weak attribute causes the declaration to be emitted as a weak symbol rather than a global. This is primarily useful in defining library functions which can be overridden in user code, though it can also be used with non-function declarations. Weak symbols are supported for ELF targets, and also for a.out targets when using the GNU assembler and linker.
Yet I use weak attribute on the function definition not the declaration.
So the question is why weak doesn't work with *.a files? Is usage of weak attribute on a definition instead of a declaration correct?
UPDATE
It has dawned on me that weak attribute used with foo() method definition had no impact on the symbol resolution. Without the attribute final binary generates the same:
1) g++ main.cpp a.a b.a prints a.c
2) g++ main.cpp b.a a.a prints b.c
So simply the first definition of the symbol is used and this is consisten with default gcc behaviour. Even though nm a.a shows that a weak symbol was emitted, it doesn't seem to affect static linking.
Is it possible to use weak attribute with static linking at all?
DESCRIPTION OF THE PROBLEM I WANT TO SOLVE
I have a library that is used by >20 clients, let's call it library A. I also provide a library B which contains testing utils for A. Somehow I need to know that library A is used in testing mode, so the simplest solution seems to be replacing a symbol during linking with B (because clients are already linking with B).
I know there are cleaner solutions to this problem, however I absolutely can't impact clients' code or their build scripts (adding parameter that would indicate testing for A or some DEFINE for compilation is out of option).
To explain what's going on here, let's talk first about your original source files, with
a.h (1):
void foo() __attribute__((weak));
and:
a.c (1):
#include "a.h"
#include <stdio.h>
void foo() { printf("%s\n", __FILE__); }
The mixture of .c and .cpp files in your sample code is irrelevant to the
issues, and all the code is C, so we'll say that main.cpp is main.c and
do all compiling and linking with gcc:
$ gcc -Wall -c main.c a.c b.c
ar rcs a.a a.o
ar rcs b.a b.o
First let's review the differences between a weakly declared symbol, like
your:
void foo() __attribute__((weak));
and a strongly declared symbol, like
void foo();
which is the default:
When a weak reference to foo (i.e. a reference to weakly declared foo) is linked in a program, the
linker need not find a definition of foo anywhere in the linkage: it may remain
undefined. If a strong reference to foo is linked in a program,
the linker needs to find a definition of foo.
A linkage may contain at most one strong definition of foo (i.e. a definition
of foo that declares it strongly). Otherwise a multiple-definition error results.
But it may contain multiple weak definitions of foo without error.
If a linkage contains one or more weak definitions of foo and also a strong
definition, then the linker chooses the strong definition and ignores the weak
ones.
If a linkage contains just one weak definition of foo and no strong
definition, inevitably the linker uses the one weak definition.
If a linkage contains multiple weak definitions of foo and no strong
definition, then the linker chooses one of the weak definitions arbitrarily.
Next, let's review the differences between inputting an object file in a linkage
and inputting a static library.
A static library is merely an ar archive of object files that we may offer to
the linker from which to select the ones it needs to carry on the linkage.
When an object file is input to a linkage, the linker unconditionally links it
into the output file.
When static library is input to a linkage, the linker examines the archive to
find any object files within it that provide definitions it needs for unresolved symbol references
that have accrued from input files already linked. If it finds any such object files
in the archive, it extracts them and links them into the output file, exactly as
if they were individually named input files and the static library was not mentioned at all.
With these observations in mind, consider the compile-and-link command:
gcc main.c a.o b.o
Behind the scenes gcc breaks it down, as it must, into a compile-step and link
step, just as if you had run:
gcc -c main.c # compile
gcc main.o a.o b.o # link
All three object files are linked unconditionally into the (default) program ./a.out. a.o contains a
weak definition of foo, as we can see:
$ nm --defined a.o
0000000000000000 W foo
Whereas b.o contains a strong definition:
$ nm --defined b.o
0000000000000000 T foo
The linker will find both definitions and choose the strong one from b.o, as we can
also see:
$ gcc main.o a.o b.o -Wl,-trace-symbol=foo
main.o: reference to foo
a.o: definition of foo
b.o: definition of foo
$ ./a.out
b.c
Reversing the linkage order of a.o and b.o will make no difference: there's
still exactly one strong definition of foo, the one in b.o.
By contrast consider the compile-and-link command:
gcc main.cpp a.a b.a
which breaks down into:
gcc -c main.cpp # compile
gcc main.o a.a b.a # link
Here, only main.o is linked unconditionally. That puts an undefined weak reference
to foo into the linkage:
$ nm --undefined main.o
w foo
U _GLOBAL_OFFSET_TABLE_
U puts
That weak reference to foo does not need a definition. So the linker will
not attempt to find a definition that resolves it in any of the object files in either a.a or b.a and
will leave it undefined in the program, as we can see:
$ gcc main.o a.a b.a -Wl,-trace-symbol=foo
main.o: reference to foo
$ nm --undefined a.out
w __cxa_finalize##GLIBC_2.2.5
w foo
w __gmon_start__
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
U __libc_start_main##GLIBC_2.2.5
U puts##GLIBC_2.2.5
Hence:
$ ./a.out
no foo
Again, it doesn't matter if you reverse the linkage order of a.a and b.a,
but this time it is because neither of them contributes anything to the linkage.
Let's turn now to the different behavior you discovered by changing a.h and a.c
to:
a.h (2):
void foo();
a.c (2):
#include "a.h"
#include <stdio.h>
void __attribute__((weak)) foo() { printf("%s\n", __FILE__); }
Once again:
$ gcc -Wall -c main.c a.c b.c
main.c: In function ‘main’:
main.c:4:18: warning: the address of ‘foo’ will always evaluate as ‘true’ [-Waddress]
int main() { if (foo) foo(); else printf("no foo\n"); }
See that warning? main.o now contains a strongly declared reference to foo:
$ nm --undefined main.o
U foo
U _GLOBAL_OFFSET_TABLE_
so the code (when linked) must have a non-null address for foo. Proceeding:
$ ar rcs a.a a.o
$ ar rcs b.a b.o
Then try the linkage:
$ gcc main.o a.o b.o
$ ./a.out
b.c
And with the object files reversed:
$ gcc main.o b.o a.o
$ ./a.out
b.c
As before, the order makes no difference. All the object files are linked. b.o provides
a strong definition of foo, a.o provides a weak one, so b.o wins.
Next try the linkage:
$ gcc main.o a.a b.a
$ ./a.out
a.c
And with the order of the libraries reversed:
$ gcc main.o b.a a.a
$ ./a.out
b.c
That does make a difference. Why? Let's redo the linkages with diagnostics:
$ gcc main.o a.a b.a -Wl,-trace,-trace-symbol=foo
/usr/bin/x86_64-linux-gnu-ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
main.o
(a.a)a.o
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
main.o: reference to foo
a.a(a.o): definition of foo
Ignoring the default libraries, the only object files of ours that get
linked were:
main.o
(a.a)a.o
And the definition of foo was taken from the archive member a.o of a.a:
a.a(a.o): definition of foo
Reversing the library order:
$ gcc main.o b.a a.a -Wl,-trace,-trace-symbol=foo
/usr/bin/x86_64-linux-gnu-ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
main.o
(b.a)b.o
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
main.o: reference to foo
b.a(b.o): definition of foo
This time the object files linked were:
main.o
(b.a)b.o
And the definition of foo was taken from b.o in b.a:
b.a(b.o): definition of foo
In the first linkage, the linker had an unresolved strong reference to
foo for which it needed a definition when it reached a.a. So it
looked in the archive for an object file that provides a definition,
and found a.o. That definition was a weak one, but that didn't matter. No
strong definition had been seen. a.o was extracted from a.a and linked,
and the reference to foo was thus resolved. Next b.a was reached, where
a strong definition of foo would have been found in b.o, if the linker still needed one
and looked for it. But it didn't need one any more and didn't look. The linkage:
gcc main.o a.a b.a
is exactly the same as:
gcc main.o a.o
And likewise the linkage:
$ gcc main.o b.a a.a
is exactly the same as:
$ gcc main.o b.o
Your real problem...
... emerges in one of your comments to the post:
I want to override [the] original function implementation when linking with a testing framework.
You want to link a program inputting some static library lib1.a
which has some member file1.o that defines a symbol foo, and you want to knock out
that definition of foo and link a different one that is defined in some other object
file file2.o.
__attribute__((weak)) isn't applicable to that problem. The solution is more
elementary. You just make sure to input file2.o to the linkage before you input
lib1.a (and before any other input that provides a definition of foo).
Then the linker will resolve references to foo with the definition provided in file2.o and will not try to find any other
definition when it reaches lib1.a. The linker will not consume lib1.a(file1.o) at all. It might as well not exist.
And what if you have put file2.o in another static library lib2.a? Then inputting
lib2.a before lib1.a will do the job of linking lib2.a(file2.o) before
lib1.a is reached and resolving foo to the definition in file2.o.
Likewise, of course, every definition provided by members of lib2.a will be linked in
preference to a definition of the same symbol provided in lib1.a. If that's not what
you want, then don't like lib2.a: link file2.o itself.
Finally
Is it possible to use [the] weak attribute with static linking at all?
Certainly. Here is a first-principles use-case:
foo.h (1)
#ifndef FOO_H
#define FOO_H
int __attribute__((weak)) foo(int i)
{
return i != 0;
}
#endif
aa.c
#include "foo.h"
int a(void)
{
return foo(0);
}
bb.c
#include "foo.h"
int b(void)
{
return foo(42);
}
prog.c
#include <stdio.h>
extern int a(void);
extern int b(void);
int main(void)
{
puts(a() ? "true" : "false");
puts(b() ? "true" : "false");
return 0;
}
Compile all the source files, requesting a seperate ELF section for each function:
$ gcc -Wall -ffunction-sections -c prog.c aa.c bb.c
Note that the weak definition of foo is compiled via foo.h into both
aa.o and bb.o, as we can see:
$ nm --defined aa.o
0000000000000000 T a
0000000000000000 W foo
$ nm --defined bb.o
0000000000000000 T b
0000000000000000 W foo
Now link a program from all the object files, requesting the linker to
discard unused sections (and give us the map-file, and some diagnostics):
$ gcc prog.o aa.o bb.o -Wl,--gc-sections,-Map=mapfile,-trace,-trace-symbol=foo
/usr/bin/x86_64-linux-gnu-ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
prog.o
aa.o
bb.o
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
aa.o: definition of foo
This linkage is no different from:
$ ar rcs libaabb.a aa.o bb.o
$ gcc prog.o libaabb.a
Despite the fact that both aa.o and bb.o were loaded, and each contains
a definition of foo, no multiple-definition error results, because each definition
is weak. aa.o was loaded before bb.o and the definition of foo was linked from aa.o.
So what happened to the definition of foo in bb.o? The mapfile shows us:
mapfile (1)
...
...
Discarded input sections
...
...
.text.foo 0x0000000000000000 0x13 bb.o
...
...
The linker discarded the function section that contained the definition
in bb.o
Let's reverse the linkage order of aa.o and bb.o:
$ gcc prog.o bb.o aa.o -Wl,--gc-sections,-Map=mapfile,-trace,-trace-symbol=foo
...
prog.o
bb.o
aa.o
...
bb.o: definition of foo
And now the opposite thing happens. bb.o is loaded before aa.o. The
definition of foo is linked from bb.o and:
mapfile (2)
...
...
Discarded input sections
...
...
.text.foo 0x0000000000000000 0x13 aa.o
...
...
the definition from aa.o is chucked away.
There you see how the linker arbitrarily chooses one of multiple
weak definitions of a symbol, in the absence of a strong definition. It simply
picks the first one you give it and ignores the rest.
What we've just done here is effectively what the GCC C++ compiler does for us when we
define a global inline function. Rewrite:
foo.h (2)
#ifndef FOO_H
#define FOO_H
inline int foo(int i)
{
return i != 0;
}
#endif
Rename our source files *.c -> *.cpp; compile and link:
$ g++ -Wall -c prog.cpp aa.cpp bb.cpp
Now there is a weak definition of foo (C++ mangled) in each of aa.o and bb.o:
$ nm --defined aa.o bb.o
aa.o:
0000000000000000 T _Z1av
0000000000000000 W _Z3fooi
bb.o:
0000000000000000 T _Z1bv
0000000000000000 W _Z3fooi
The linkage uses the first definition it finds:
$ g++ prog.o aa.o bb.o -Wl,-Map=mapfile,-trace,-trace-symbol=_Z3fooi
...
prog.o
aa.o
bb.o
...
aa.o: definition of _Z3fooi
bb.o: reference to _Z3fooi
and throws away the other one:
mapfile (3)
...
...
Discarded input sections
...
...
.text._Z3fooi 0x0000000000000000 0x13 bb.o
...
...
And as you may know, every instantiation of the C++ function template in
global scope (or instantiation of a class template member function) is
an inline global function. Rewrite again:
#ifndef FOO_H
#define FOO_H
template<typename T>
T foo(T i)
{
return i != 0;
}
#endif
Recompile:
$ g++ -Wall -c prog.cpp aa.cpp bb.cpp
Again:
$ nm --defined aa.o bb.o
aa.o:
0000000000000000 T _Z1av
0000000000000000 W _Z3fooIiET_S0_
bb.o:
0000000000000000 T _Z1bv
0000000000000000 W _Z3fooIiET_S0_
each of aa.o and bb.o has a weak definition of:
$ c++filt _Z3fooIiET_S0_
int foo<int>(int)
and the linkage behaviour is now familiar. One way:
$ g++ prog.o aa.o bb.o -Wl,-Map=mapfile,-trace,-trace-symbol=_Z3fooIiET_S0_
...
prog.o
aa.o
bb.o
...
aa.o: definition of _Z3fooIiET_S0_
bb.o: reference to _Z3fooIiET_S0_
and the other way:
$ g++ prog.o bb.o aa.o -Wl,-Map=mapfile,-trace,-trace-symbol=_Z3fooIiET_S0_
...
prog.o
bb.o
aa.o
...
bb.o: definition of _Z3fooIiET_S0_
aa.o: reference to _Z3fooIiET_S0_
Our program's behavior is unchanged by the rewrites:
$ ./a.out
false
true
So the application of the weak attribute to symbols in the linkage of ELF objects -
whether static or dynamic - enables the GCC implementation of C++ templates
for the GNU linker. You could fairly say it enables the GCC implementation of modern C++.
I find that here is the best explanation:
The linker will only search through libraries to resolve a reference if it cannot resolve that reference after searching all input objects. If required, the libraries are searched from left to right according to their position on the linker command line. Objects within the library will be searched by the order in which they were archived. As soon as armlink finds a symbol match for the reference, the searching is finished, even if it matches a weak definition.
The ELF ABI section 4.6.1.2 says:
"A weak definition does not change the rules by which object files are selected from libraries. However, if a link set contains both a weak definition and a non-weak definition, the non-weak definition will always be used."
The "link set" is the set of objects that have been loaded by the linker. It does not include objects from libraries that are not required.
Therefore archiving two objects where one contains the weak definition of a given symbol and the other contains the non-weak definition of that symbol, into a library or separate libraries, is not recommended.
Observe the following. Basically renamed mv a.c definition.c mv b.c noweak.c and mv second_a.c declaration.c.
> for i in Makefile *.c; do echo "cat $i <<EOF"; cat $i; echo EOF; done
cat Makefile <<EOF
tgt=
tgt+=only_weak_1.out only_weak_2.out
tgt+=definition.out declaration.out noweak.out
tgt+=definition_static.out declaration_static.out noweak_static.out
tgt+=1.out 2.out 3.out 4.out
tgt+=5.out 6.out 7.out 8.out
tgt+=10.out 11.out 12.out
tgt+=13.out
tgt+=14.out
only_weak_1_obj= definition.o declaration.o
only_weak_2_obj= declaration.o definition.o
definition_obj= definition.o
declaration_obj= declaration.o
noweak_obj= noweak.o
definition_static_obj= definition.a
declaration_static_obj= declaration.a
noweak_static_obj= noweak.a
1_obj= declaration.o noweak.o
2_obj= noweak.o declaration.o
3_obj= declaration.a noweak.a
4_obj= noweak.a declaration.a
5_obj= definition.o noweak.o
6_obj= noweak.o definition.o
7_obj= definition.a noweak.a
8_obj= noweak.a definition.a
10_obj= noweak.a definition.a declaration.a
11_obj= definition.a declaration.a noweak.a
12_obj= declaration.a definition.a noweak.a
13_obj= all.a
14_obj= all.o
.PRECIOUS: % %.o %.c %.a
def: run
.PHONY: run
run: $(tgt)
{ $(foreach a,$^,echo "$($(a:.out=)_obj)#->#$(a)#:#$$(./$(a))";) } | { echo; column -t -s'#' -N 'objects, ,executable, ,output' -o' '; echo; }
.SECONDEXPANSION:
%.out: main.o $$(%_obj)
$(CC) -o $# $^
%.o: %.c
$(CC) -c -o $# $^
%.a: %.o
ar cr $# $^
all.a: declaration.o definition.o noweak.o
ar cr $# $^
all.o: declaration.o definition.o noweak.o
$(LD) -i -o $# $^
clean:
rm -fv *.o *.a *.out
EOF
cat declaration.c <<EOF
#include <stdio.h>
__attribute__((__weak__)) void foo();
void foo() { printf("%s\n", __FILE__); }
EOF
cat definition.c <<EOF
#include <stdio.h>
__attribute__((__weak__)) void foo() { printf("%s\n", __FILE__); }
EOF
cat main.c <<EOF
#include <stdio.h>
void foo();
int main() {
if (foo) foo(); else printf("no foo\n");
return 0;
}
EOF
cat noweak.c <<EOF
#include <stdio.h>
void foo() { printf("%s\n", __FILE__); }
EOF
> make
cc -c -o definition.o definition.c
cc -c -o declaration.o declaration.c
cc -c -o main.o main.c
cc -o only_weak_1.out main.o definition.o declaration.o
cc -o only_weak_2.out main.o declaration.o definition.o
cc -o definition.out main.o definition.o
cc -o declaration.out main.o declaration.o
cc -c -o noweak.o noweak.c
cc -o noweak.out main.o noweak.o
ar cr definition.a definition.o
cc -o definition_static.out main.o definition.a
ar cr declaration.a declaration.o
cc -o declaration_static.out main.o declaration.a
ar cr noweak.a noweak.o
cc -o noweak_static.out main.o noweak.a
cc -o 1.out main.o declaration.o noweak.o
cc -o 2.out main.o noweak.o declaration.o
cc -o 3.out main.o declaration.a noweak.a
cc -o 4.out main.o noweak.a declaration.a
cc -o 5.out main.o definition.o noweak.o
cc -o 6.out main.o noweak.o definition.o
cc -o 7.out main.o definition.a noweak.a
cc -o 8.out main.o noweak.a definition.a
cc -o 10.out main.o noweak.a definition.a declaration.a
cc -o 11.out main.o definition.a declaration.a noweak.a
cc -o 12.out main.o declaration.a definition.a noweak.a
ar cr all.a declaration.o definition.o noweak.o
cc -o 13.out main.o all.a
ld -i -o all.o declaration.o definition.o noweak.o
cc -o 14.out main.o all.o
{ echo "definition.o declaration.o#->#only_weak_1.out#:#$(./only_weak_1.out)"; echo "declaration.o definition.o#->#only_weak_2.out#:#$(./only_weak_2.out)"; echo "definition.o#->#definition.out#:#$(./definition.out)"; echo "declaration.o#->#declaration.out#:#$(./declaration.out)"; echo "noweak.o#->#noweak.out#:#$(./noweak.out)"; echo "definition.a#->#definition_static.out#:#$(./definition_static.out)"; echo "declaration.a#->#declaration_static.out#:#$(./declaration_static.out)"; echo "noweak.a#->#noweak_static.out#:#$(./noweak_static.out)"; echo "declaration.o noweak.o#->#1.out#:#$(./1.out)"; echo "noweak.o declaration.o#->#2.out#:#$(./2.out)"; echo "declaration.a noweak.a#->#3.out#:#$(./3.out)"; echo "noweak.a declaration.a#->#4.out#:#$(./4.out)"; echo "definition.o noweak.o#->#5.out#:#$(./5.out)"; echo "noweak.o definition.o#->#6.out#:#$(./6.out)"; echo "definition.a noweak.a#->#7.out#:#$(./7.out)"; echo "noweak.a definition.a#->#8.out#:#$(./8.out)"; echo "noweak.a definition.a declaration.a#->#10.out#:#$(./10.out)"; echo "definition.a declaration.a noweak.a#->#11.out#:#$(./11.out)"; echo "declaration.a definition.a noweak.a#->#12.out#:#$(./12.out)"; echo "all.a#->#13.out#:#$(./13.out)"; echo "all.o#->#14.out#:#$(./14.out)"; } | { echo; column -t -s'#' -N 'objects, ,executable, ,output' -o' '; echo; }
objects executable output
definition.o declaration.o -> only_weak_1.out : definition.c
declaration.o definition.o -> only_weak_2.out : declaration.c
definition.o -> definition.out : definition.c
declaration.o -> declaration.out : declaration.c
noweak.o -> noweak.out : noweak.c
definition.a -> definition_static.out : definition.c
declaration.a -> declaration_static.out : declaration.c
noweak.a -> noweak_static.out : noweak.c
declaration.o noweak.o -> 1.out : noweak.c
noweak.o declaration.o -> 2.out : noweak.c
declaration.a noweak.a -> 3.out : declaration.c
noweak.a declaration.a -> 4.out : noweak.c
definition.o noweak.o -> 5.out : noweak.c
noweak.o definition.o -> 6.out : noweak.c
definition.a noweak.a -> 7.out : definition.c
noweak.a definition.a -> 8.out : noweak.c
noweak.a definition.a declaration.a -> 10.out : noweak.c
definition.a declaration.a noweak.a -> 11.out : definition.c
declaration.a definition.a noweak.a -> 12.out : declaration.c
all.a -> 13.out : declaration.c
all.o -> 14.out : noweak.c
In case only weak symbols are used (case only_weak_1 and only_weak_2) the first definition is used.
In case of only static libraries (case 3, 4, 7, 8, 10, 11, 12, 13) the first definition is used.
In case only object files are used (cases 1, 2, 5, 6, 14) the weak symbols are omitted and only the symbol from noweak is used.
From the link I provided:
There are different ways to guarantee armlink selecting a non-weak version of a given symbol:
- Do not archive such objects
- Ensure that the weak and non-weak symbols are contained within the same object before archiving
- Use partial linking as an alternative.
Related
I have some static library that I can't change or rebuild. The library uses global variables. Something like this:
//lib A
#include <iostream>
static int i = 0;
void printA(){
std::cout << i++ << std::endl;
}
I want to create two shared libraries that have their own "copy" of static library and its global state:
//lib B
#include "liba.h"
void printB(){
printA();
}
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
//lib C
#include "liba.h"
void printC(){
printA();
}
... and use them simultaneously:
#include "libb.h"
#include "libc.h"
int main(){
printB();
printB();
printC();
printC();
}
I expect following output:
0
1
0
1
.. but actually get:
0
1
2
3
Seems like libB and libC share common counter variable. If had access to libA source code, I would rebuild it with -fvisibility=hidden. But unfortunately I have only binary.
Is there any way to achieve expected behavior without libA rebuilding?
If LibA uses a static counter which libB and libC increment by calling printA, then there's no way to do what you want without object file manipulation or non-portable hacks.
The linker resolves all references to global variables (even statics) to the same symbol at link time.
If you're willing to manipulate object files, then the following should work for you:
$ objcopy --prefix-symbols=copy_ liba.a liba-copy.a
#define printA copy_ printA
#include "liba.h"
/* ... */
If you can get the symbols from the static library using nm (the name you'll be looking for will be in the form of <counter name>.<process ID>) and you do something like the following, then you can read and write the static counter variable at runtime:
int counter asm("<counter name>.<process ID>");
counter = 0;
Note that this process will have to be repeated after every update of the library.
You can copy the static library and rename all symbols that use the global state. Because the symbols are compiled with c++, you are out of luck, the symbols are mangled.
You can write a C interface for all the accesses and recompile the static library hiding it's symbols and then use some objcopy --prefix-symbols org++ -Wl,--wrap=printA to prefix/rename the C symbols.
Or you need to know beforehand the already mangled C++ names, and then call objcopy --redefine-sym _Z6printAv=_Z10printAcopyv etc. for each symbol the library exports.
Below is the test setup that calls the objcopy on the mangled names. I found out the symbol names by inspecting the object files, nm a.o and nm c.o. Here it is:
cat <<EOF >Makefile
all: liba.a b.o main.o c.o
# we have access only to liba.a only
objcopy --redefine-sym _Z6printAv=_Z10printAcopyv liba.a libacopy.a
g++ main.o b.o c.o liba.a libacopy.a -o a.out
./a.out
liba.a: a.o
ar rcs liba.a a.o
clean:
rm -fr *.o *.a *.out tmp
EOF
cat <<EOF >a.cpp
#include <iostream>
static int i = 0;
void printA(){
std::cout << i++ << std::endl;
}
EOF
cat <<EOF >b.cpp
void printA();
void printB(){
printA();
}
EOF
cat <<EOF >c.cpp
void printAcopy();
void printC(){
printAcopy();
}
EOF
cat <<EOF >main.cpp
void printB();
void printC();
int main(){
printB();
printB();
printC();
printC();
}
EOF
You can compile with make and run:
g++ -c -o a.o a.cpp
ar rcs liba.a a.o
g++ -c -o b.o b.cpp
g++ -c -o main.o main.cpp
g++ -c -o c.o c.cpp
# we have access only to liba.a only
objcopy --redefine-sym _Z6printAv=_Z10printAcopyv liba.a libacopy.a
g++ main.o b.o c.o liba.a libacopy.a -o a.out
./a.out
0
1
0
1
lib.h:
#include <iostream>
namespace lib {
template <class T>
void f(T t)
{
std::cout << "lib f " << t << std::endl;
}
}
client.cpp:
#include "lib.h"
// explicit instantiation
template
void lib::f(char);
int main()
{
lib::f('x');
}
libmock.h:
#include <iostream>
#include "lib.h"
namespace lib {
template <>
void f(char c)
{
std::cout << "libmock f " << c << std::endl;
}
}
Makefile:
run: prod test
./prod
./test
prod: client.o
${CXX} -o $# $^
test: client.o libmock.o
${CXX} -o $# $^
clean:
-rm *.o prod test
Using GCC 4.3.2 (and also "IBM XL C/C++ for AIX, V11.1 (5724-X13)"), I get the results that I expect:
$ make
g++ -c -o client.o client.cpp
g++ -o prod client.o
g++ -c -o libmock.o libmock.cpp
g++ -o test client.o libmock.o
./prod
lib f x
./test
libmock f x
That is, I've injected new functionality into the client by linking it with an object that provides a more-specialized function template than the one offered by the library.
However, if I use "CC: Sun C++ 5.12 SunOS_sparc Patch 148506-14 2013/09/24", then I get this error:
$ CXX=CC make
CC -c -o client.o client.cpp
CC -o prod client.o
CC -c -o libmock.o libmock.cpp
CC -o test client.o libmock.o
ld: fatal: symbol 'void lib::f<char>(__type_0)' is multiply-defined:
(file client.o type=FUNC; file libmock.o type=FUNC);
Makefile:9: recipe for target 'test' failed
make: *** [test] Error 2
My solution must work with all three of these compilers. Am I just getting lucky with some undefined behavior in GCC and AIX? Are there some options I could pass to the Sun compiler to get this to work? Does what I'm trying to do show that I'm not fully understanding these template concepts? Enlighten me, please!
Your test binary that links libmock.o and client.o together violates the one definition rule (in the client translation unit it uses the default version and in the libmock translation unit it uses the specialized version) and thus both linker behaviors are ok.
I will continue thinking about alternatives but right now the only solutiion I can think of is to conditionally include libmock.h in client.cpp based on whether you're doing the mock test build or not.
I did a sample project in linux but i am getting error while running main Makefile
Project Info:
project/database folder having files database.h , database.cpp , bulid-database ,Makefile
database.h
/*data base file*/
#include<iostream>
using namespace std;
class mydatabase
{
public:
mydatabase(int a , int b);
int sum(){return m_a +m_b;}
int diff(){return m_a -m_b;}
int mul(){return m_a *m_b;}
float div(){return m_a /m_b;}
int reminder(){return m_a %m_b;}
private:
int m_a , m_b;
};
database.cpp
#include "database.h"
mydatabase::mydatabase(int a ,int b):m_a(a) , m_b(b)
{
}
bulid-database
make
if [ -f libdatabase.a ];
then
echo "Database-Library Build Success"
cp libdatabase.a ../LIBs/
else
echo "databse-Library Build Failure"
fi
Makefile
HEADERFILES = $(wildcard *.h)
CPPFILES = $(wildcard *.cpp)
OBJFILES = $(patsubst %.cpp,%.o ,$(wildcard *.cpp))
$(OBJFILES): %.o : %.cpp $(HEADERFILES)
g++ -c -o $# $<
ar ruv libdatabase.a $#
ranlib libdatabase.a
project/Main folder having files main.cpp , Makefile
main.cpp
#include "database.h"
#include <iostream>
int main()
{
mydatabase *obj = new mydatabase(10 ,5);
std::cout<<"sum is"<<obj->sum()<<endl;
std::cout<<"diff is"<<obj->diff()<<endl;
std::cout<<"mul is"<<obj->mul()<<endl;
std::cout<<"div is"<<obj->div()<<endl;
std::cout<<"reminder is"<<obj->reminder()<<endl;
getchar();
return 0;
}
Makefile
CC = g++
INCPATH = -I. \
-I.. \
-I../database
LIBPATH = -L../LIBs
LDFLAGS = ${LIBPATH}/libdatabase.a
CFLAGS = ${INCPATH}
testdate:main.o
$(CC) $(CFLAGS) -o testdate main.o $(LDFLAGS)
main.o:main.cpp
$(CC) $(CFLAGS) -c -o main.o main.cpp
ISSUE: database make file is working fine but main Makefile i am having some issue like
Error: main.o: In function main':
main.cpp:(.text+0x92): undefined reference tomydatabase::mydatabase(int, int)'
collect2: ld returned 1 exit status
This line is wrong:
$(CC) $(CFLAGS) -o testdate $(LDFLAGS) main.o
because the library should be specificed AFTER the object main.o on the line. This is due to the way the linker handles the objects. Look at this example:
gcc -o test someobject.o library.a
The linker will:
look up all undefined references of someobject.o and store them
then it opens library.a and resolves the undefined references via library.a
then it closes library.a
If the object and the library are in the other way around, then the linker opens library.a, sees no undefined references in its table and closes it. Then it tries and compiles someobject.o and the undefined references are never satisfied
EDIT:
This is a well-known caveat of GCC, a more detailed stack-overflow explanation can be seen here, and options --start-group and --end-group can help resolve cases where A depends on B, and B depends on A.
It's your Makefile. You want:
libdatabase.a
or
-ldatabase
at the end of your main compile line
This is a scaled down version of a problem I am facing with clang++ on Mac OS X. This was seriously edited to better reflect the genuine problem (the first attempt to describe the issue was not exhibiting the problem).
The failure
I have this big piece of software in C++ with a large set of symbols in the object files, so I'm using -fvisibility=hidden to keep my symbol tables small. It is well known that in such a case one must pay extra attention to the vtables, and I suppose I face this problem. I don't know however, how to address it elegantly in a way that pleases both gcc and clang.
Consider a base class which features a down-casting operator, as, and a derived class template, that contains some payload. The pair base/derived<T> is used to implement type-erasure:
// foo.hh
#define API __attribute__((visibility("default")))
struct API base
{
virtual ~base() {}
template <typename T>
const T& as() const
{
return dynamic_cast<const T&>(*this);
}
};
template <typename T>
struct API derived: base
{};
struct payload {}; // *not* flagged as "default visibility".
API void bar(const base& b);
API void baz(const base& b);
Then I have two different compilation units that provide a similar service, which I can approximate as twice the same feature: down-casting from base to derive<payload>:
// bar.cc
#include "foo.hh"
void bar(const base& b)
{
b.as<derived<payload>>();
}
and
// baz.cc
#include "foo.hh"
void baz(const base& b)
{
b.as<derived<payload>>();
}
From these two files, I build a dylib. Here is the main function, calling these functions from the dylib:
// main.cc
#include <stdexcept>
#include <iostream>
#include "foo.hh"
int main()
try
{
derived<payload> d;
bar(d);
baz(d);
}
catch (std::exception& e)
{
std::cerr << e.what() << std::endl;
}
Finally, a Makefile to compile and link everybody. Nothing special here, except, of course, -fvisibility=hidden.
CXX = clang++
CXXFLAGS = -std=c++11 -fvisibility=hidden
all: main
main: main.o bar.dylib baz.dylib
$(CXX) -o $# $^
%.dylib: %.cc foo.hh
$(CXX) $(CXXFLAGS) -shared -o $# $<
%.o: %.cc foo.hh
$(CXX) $(CXXFLAGS) -c -o $# $<
clean:
rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
The run succeeds with gcc (4.8) on OS X:
$ make clean && make CXX=g++-mp-4.8 && ./main
rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
g++-mp-4.8 -std=c++11 -fvisibility=hidden -c main.cc -o main.o
g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
g++-mp-4.8 -o main main.o bar.dylib baz.dylib
However with clang (3.4), this fails:
$ make clean && make CXX=clang++-mp-3.4 && ./main
rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -c main.cc -o main.o
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
clang++-mp-3.4 -o main main.o bar.dylib baz.dylib
std::bad_cast
However it works if I use
struct API payload {};
but I do not want to expose the payload type. So my questions are:
why are GCC and Clang different here?
is it really working with GCC, or I was just "lucky" in my use of undefined behavior?
do I have a means to avoid making payload go public with Clang++?
Thanks in advance.
Type equality of visible class templates with invisible type parameters (EDIT)
I have now a better understanding of what is happening. It is appears that both GCC and clang require both the class template and its parameter to be visible (in the ELF sense) to build a unique type. If you change the bar.cc and baz.cc functions as follows:
// bar.cc
#include "foo.hh"
void bar(const base& b)
{
std::cerr
<< "bar value: " << &typeid(b) << std::endl
<< "bar type: " << &typeid(derived<payload>) << std::endl
<< "bar equal: " << (typeid(b) == typeid(derived<payload>)) << std::endl;
b.as<derived<payload>>();
}
and if you make payload visible too:
struct API payload {};
then you will see that both GCC and Clang will succeed:
$ make clean && make CXX=g++-mp-4.8
rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
g++-mp-4.8 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
./g++-mp-4.8 -o main main.o bar.dylib baz.dylib
$ ./main
bar value: 0x106785140
bar type: 0x106785140
bar equal: 1
baz value: 0x106785140
baz type: 0x106785140
baz equal: 1
$ make clean && make CXX=clang++-mp-3.4
rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
clang++-mp-3.4 -o main main.o bar.dylib baz.dylib
$ ./main
bar value: 0x10a6d5110
bar type: 0x10a6d5110
bar equal: 1
baz value: 0x10a6d5110
baz type: 0x10a6d5110
baz equal: 1
Type equality is easy to check, there is actually a single instantiation of the type, as witnessed by its unique address.
However, if you remove the visible attribute from payload:
struct payload {};
then you get with GCC:
$ make clean && make CXX=g++-mp-4.8
rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
g++-mp-4.8 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
g++-mp-4.8 -o main main.o bar.dylib baz.dylib
$ ./main
bar value: 0x10faea120
bar type: 0x10faf1090
bar equal: 1
baz value: 0x10faea120
baz type: 0x10fafb090
baz equal: 1
Now there are several instantiation of the type derived<payload> (as witnessed by the three different addresses), but GCC sees these types are equal, and (of course) the two dynamic_cast pass.
In the case of clang, it's different:
$ make clean && make CXX=clang++-mp-3.4
rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
.clang++-mp-3.4 -o main main.o bar.dylib baz.dylib
$ ./main
bar value: 0x1012ae0f0
bar type: 0x1012b3090
bar equal: 0
std::bad_cast
There are also three instantiations of the type (removing the failing dynamic_cast does show that there are three), but this time, they are not equal, and the dynamic_cast (of course) fails.
Now the question turns into:
1. is this difference between both compilers wanted by their authors
2. if not, what is "expected" behavior between both
I prefer GCC's semantics, as it allows to really implement type-erasure without any need to expose publicly the wrapped types.
I had reported this to the people from LLVM, and it was first noted that if it works in the case of GCC, it's because:
I think the difference is actually in the c++ library. It looks like
libstdc++ changed to always use strcmp of the typeinfo names:
https://gcc.gnu.org/viewcvs/gcc?view=revision&revision=149964
Should we do the same with libc++?
To this, it was clearly answered that:
No. It pessimizes correctly behaving code to work around code that
violates the ELF ABI. Consider an application that loads plugins with
RTLD_LOCAL. Two plugins implement a (hidden) type called "Plugin". The
GCC change now makes this completely separate types identical for all
RTTI purposes. That makes no sense at all.
So I can't do what I want with Clang: reduce the number of published symbols. But it appears to be saner than the current behavior of GCC. Too bad.
I've run into this problem recently, and #akim (the OP) has diagnosed it.
A workaround is to write your own dynamic_cast_to_private_exact_type<T> or somesuch that checks the typeid's string name.
template<class T>
struct dynamic_cast_to_exact_type_helper;
template<class T>
struct dynamic_cast_to_exact_type_helper<T*>
{
template<class U>
T* operator()(U* u) const {
if (!u) return nullptr;
auto const& uid = typeid(*u);
auto const& tid = typeid(T);
if (uid == tid) return static_cast<T*>(u); // shortcut
if (uid.hash_code() != tid.hash_code()) return nullptr; // hash compare to reject faster
if (uid.name() == tid.name()) return static_cast<T*>(u); // compare names
return nullptr;
}
};
template<class T>
struct dynamic_cast_to_exact_type_helper<T&>
{
template<class U>
T& operator()(U& u) const {
T* r = dynamic_cast_to_exact_type<T&>{}(std::addressof(u));
if (!r) throw std::bad_cast{};
return *r;
}
}
template<class T, class U>
T dynamic_cast_to_exact_type( U&& u ) {
return dynamic_cast_to_exact_type_helper<T>{}( std::forward<U>(u) );
}
Note that this can have false positives, if two modules have a different Foo type that is unrelated. Modules should put their private types in anonymous namespaces to avoid this.
I don't know how to similarly handle intermediate types, as we can only check the exact type in a typeid comparsion and cannot iterate over the type inheritance tree.
I have a question about making static libraries that use other static libraries.
I set up an example with 3 files - main.cpp, slib1.cpp and slib2.cpp. slib1.cpp and slib2.cpp are both compiled as individual static libraries (e.g. I end up with slib1.a and slib2.a) main.cpp is compiled into a standard ELF executable linked against both libraries.
There also exists a header file named main.h which prototypes the functions in slib1 and slib2.
main.cpp calls a function called lib2func() from slib2. This function in turn calls lib1func() from slib1.
If I compile the code as is, g++ will return with a linker error stating that it could not find lib1func() in slib1. However, if I make a call to lib1func() BEFORE any calls to any functions in slib2, the code compiles and works correctly.
My question is simply as follows: is it possible to create a static library that depends on another static library? It would seem like a very severe limitation if this were not possible.
The source code for this problem is attached below:
main.h:
#ifndef MAIN_H
#define MAIN_H
int lib1func();
int lib2func();
#endif
slib1.cpp:
#include "main.h"
int lib1func() {
return 1;
}
slib2.cpp:
#include "main.h"
int lib2func() {
return lib1func();
}
main.cpp:
#include <iostream>
#include "main.h"
int main(int argc, char **argv) {
//lib1func(); // Uncomment and compile will succeed. WHY??
cout << "Ans: " << lib2func() << endl;
return 0;
}
gcc output (with line commented out):
g++ -o src/slib1.o -c src/slib1.cpp
ar rc libslib1.a src/slib1.o
ranlib libslib1.a
g++ -o src/slib2.o -c src/slib2.cpp
ar rc libslib2.a src/slib2.o
ranlib libslib2.a
g++ -o src/main.o -c src/main.cpp
g++ -o main src/main.o -L. -lslib1 -lslib2
./libslib2.a(slib2.o): In function `lib2func()':
slib2.cpp:(.text+0x5): undefined reference to `lib1func()'
collect2: ld returned 1 exit status
gcc output (with line uncommented)
g++ -o src/slib1.o -c src/slib1.cpp
ar rc libslib1.a src/slib1.o
ranlib libslib1.a
g++ -o src/slib2.o -c src/slib2.cpp
ar rc libslib2.a src/slib2.o
ranlib libslib2.a
g++ -o src/main.o -c src/main.cpp
g++ -o main src/main.o -L. -lslib1 -lslib2
$ ./main
Ans: 1
Please, try g++ -o main src/main.o -L. -Wl,--start-group -lslib1 -lslib2 -Wl,--end-group.
Group defined with --start-group, --end-group helps to resolve circular dependencies between libraries.
See also: GCC: what are the --start-group and --end-group command line options?
The order make the difference. Here's from gcc(1) manual page:
It makes a difference where in the command you write this option; the linker searches and processes libraries and object files in the order they are specified. Thus, foo.o -lz bar.o searches library z after file foo.o but before bar.o. If bar.o refers to functions in z, those functions may not be loaded.