This question already has an answer here:
How to include all objects of an archive in a shared object?
(1 answer)
Closed 10 months ago.
I'm compiling some shared objects file into an archive.a:
$ g++ -c -Iinclude/ -fPIC -O0 -o object1.o source1.cpp
$ g++ -c -Iinclude/ -fPIC -O0 -o object2.o source2.cpp
$ ar rvs archive.a object1.o object2.o
r - object1.o
r - object2.o
So far so good. The resulting archive.a has a good size of some KB. A dump with nm shows that the corresponding object-files are contained within the files.
Now I'm wanting to compile several of these archives into a shared object file.
g++ -g -O0 -Iinclude/ -I/usr/include/somelibrary -shared -o libLibrary.so archive1.a archive2.a
The result is that my resulting library file is nearly empty:
$ nm -D libLibrary.so
w _Jv_RegisterClasses
0000000000201010 A __bss_start
w __cxa_finalize
w __gmon_start__
0000000000201010 A _edata
0000000000201020 A _end
0000000000000578 T _fini
0000000000000430 T _init
Any idea what I'm doing wrong?
Edit:
When I try the switch -Wl,--whole-archive, following happens:
/usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS): In function `__libc_csu_init':
(.text+0xd): undefined reference to `__init_array_end'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS): relocation R_X86_64_PC32 against undefined hidden symbol `__init_array_end' can not be used when making a shared object
/usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
make: *** [libKeynect.so] Error 1
symbols/object files in .a files that's not used, will be discarded by the linker.
Use -Wl,--whole-archive for the linking to include the entire .a file
Edit, you'll need to add -Wl,--no-whole-archive after you specify your library as well, so the whole thing will be -Wl,--whole-archive archive1.a archive2.a -Wl,--no-whole-archive
Regarding your edit: Put "-Wl,--no-whole-archive" at the end of the link command you're running. That fixed it for me.
Related
Working on embedded device (ARM, uClibc), I have a static executable which statically linked with different libraries, and have dynamic loading feature using dlopen.
set(EXTERNAL_LIBS "-lpthread -lpcap -lcurl -ldl")
target_link_libraries(myApp -static ${EXTERNAL_LIBS})
When loading simple plugin everything works fine.
void plugin::execute() {
std::cout << "hello world" << std::endl;
}
When adding string variable:
void plugin::execute() {
//THIS IS NOT WORKING
std::string test = "hello world from thing";
std::cout << test << std::endl;
}
I get:
"can't resolve symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_'"
I've tried adding -rdynamic as suggested here:
dlopen a dynamic library from a static library, when the dynamic library uses symbols of the static one
by adding:
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -rdynamic -Wl,-export-dynamic")
But it still doesn't work.
The missing symbol DOES exists in the static binary (verified using nm)
What am I missing here??
Added simplified output of the build process:
Compiling object files
arm-linux-uclibcgnueabi-g++ -fPIC -std=gnu++98 -o CMakeFiles/libstaticlib.dir/test1.cpp.o -c /work/src/test1.cpp
arm-linux-uclibcgnueabi-gcc -fPIC -std=gnu++98 -o CMakeFiles/libstaticlib.dir/test2.cpp.o -c /work/src/test2.cpp
Linking CXX static library
arm-linux-uclibcgnueabi-ar qc libstaticlib.a CMakeFiles/libstaticlib.dir/test1.cpp.o CMakeFiles/libstaticlib.dir/test2.cpp.o
arm-linux-uclibcgnueabi-ranlib libstaticlib.a
Compiling myApp
arm-linux-uclibcgnueabi-g++ -fPIE -std=gnu++98 -o CMakeFiles/myapp.dir/main.cpp.o -c /work/src/main.cpp
Linking CXX executable
arm-linux-uclibcgnueabi-g++ -rdynamic CMakeFiles/myapp.dir/main.cpp.o -o myapp -L/work/lib -Wl,-rpath,/work/lib -rdynamic -static libstaticlib.a -lpthread -lpcap -lcurl -ldl
Compiling plugin
arm-linux-uclibcgnueabi-g++ -fPIC -std=gnu++98 -o CMakeFiles/plugin.dir/plugin/plugin.cpp.o -c /work/src/plugins/plugin/plugin.cpp
Linking CXX shared library ../libplugin.so
arm-linux-uclibcgnueabi-g++ -fPIC -shared -Wl,-soname,libplugin.so -o ../libplugin.so CMakeFiles/plugin.dir/plugin/plugin.cpp.o -L/work/lib
output of readelf -s myapp | grep ...:
0021ce74 68 FUNC WEAK DEFAULT 2 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_
-rdynamic is a GCC linkage option. So you can pass it directly to
GCC when it invokes the linker (ld). The effect of -rdynamic is to
make GCC pass --export-dynamic in its invocation of ld, as you may see
in the GCC manual: 3.14 Options for Linking
--export-dynamic is not a GCC option, but is an ld option. You
can tell GCC to pass this option when it invokes ld by passing -Wl,--export-dynamic
to GCC.
So your GCC options:
-rdynamic -Wl,-export-dynamic
do the same thing twice: -rdynamic would be enough.
But the setting:
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -rdynamic")
will not cause GCC to pass -rdynamic when it invokes the linker.
That is because CMAKE_CXX_FLAGS sets the options that will be passed to
each C++ compilation. Since no linkage happens in compilation, linkage
options are ignored and have no effect. Linkage options should be set in
CMAKE_EXE_LINKER_FLAGS,
like:
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -rdynamic")
But even then...
the title of your question will remain true, because -rdynamic doesn't work from a static binary, period.
From the linker man page
--export-dynamic
When creating a dynamically linked executable, using the -E option or the
--export-dynamic option causes the linker to add all symbols to the dynamic symbol table.
My emphasis. And you are not creating a dynamically linked executable, because you
are linking -static. There will be no dynamic symbol table to which all symbols
could be added.
Here's an elementary demonstration.
main.c
int foo() {
return 0;
}
int main()
{
return foo();
}
Compile and link normally:
$ gcc main.c
Dynamic symbol table:
$ nm -D a.out
w __cxa_finalize
w __gmon_start__
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
U __libc_start_main
Compile and link -rdynamic; see dynamic symbol table:
$ gcc -rdynamic main.c
$ nm -D a.out
0000000000201010 B __bss_start
w __cxa_finalize
0000000000201000 D __data_start
0000000000201000 W data_start
0000000000201010 D _edata
0000000000201018 B _end
0000000000000884 T _fini
00000000000007ea T foo
w __gmon_start__
00000000000006a0 T _init
0000000000000890 R _IO_stdin_used
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
0000000000000880 T __libc_csu_fini
0000000000000810 T __libc_csu_init
U __libc_start_main
00000000000007f5 T main
00000000000006e0 T _start
More symbols now, including main and foo.
Compile and link -static; see dynamic symbol table:
$ gcc -static main.c
$ nm -D a.out
nm: a.out: no symbols
And finally:
$ gcc -rdynamic -static main.c
$ nm -D a.out
nm: a.out: no symbols
You just cannot link a program statically if you want a plugin to reference symbols that it defines.
I'm trying to build some example c++ code that use boost library. I use this as reference example of static linking.
And everything is fine when I build with dynamic libs.
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/main.o src/main.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/ThreadExample.o src/ThreadExample.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/Utils.o src/Utils.cpp
g++ src/main.o src/ThreadExample.o src/Utils.o -lboost_thread -lboost_filesystem -lboost_system -lboost_timer -o ThreadExampleBinary
But when I use static libs I get lots of undefined reference errors:
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/main.o src/main.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/ThreadExample.o src/ThreadExample.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/Utils.o src/Utils.cpp
g++ -static src/main.o src/ThreadExample.o src/Utils.o -lboost_thread -lboost_filesystem -lboost_system -lboost_timer -o ThreadExampleBinary
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::start()':
(.text+0x7fd): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::stop()':
(.text+0x94c): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::elapsed() const':
(.text+0xa59): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::resume()':
(.text+0xb60): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::auto_cpu_timer::auto_cpu_timer(std::ostream&, short)':
(.text+0xca5): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o):(.text+0xd4e): more undefined references to `boost::chrono::steady_clock::now()' follow
collect2: error: ld returned 1 exit status
make: *** [ThreadExampleBinary] Error 1
Seems this can be fixed adding additional -lboost_chrono library.
But why it works in dinamic setting?
With static linking, you have to also statically link to any libraries depended on by the libraries you are linking to.
The difference is that shared libraries have an entry in the ELF header, called NEEDED that lists other shared libraries that are to be included when you link in this one.
You can see them with this command:
$ objdump -p /usr/lib/libboost_timer.so | grep NEEDED
NEEDED libboost_chrono.so.1.60.0
NEEDED libboost_system.so.1.60.0
NEEDED librt.so.1
NEEDED libstdc++.so.6
NEEDED libgcc_s.so.1
NEEDED libc.so.6
But for static libraries there is no such system, as they are simply collection of object files.
It is worth noting that the NEEDED entry in the shared objects are entirely optional, and if they are not available, then they will behave exactly like the static ones. But most shared libraries include them.
Many libraries use the pkg-config infrastructure to provide the full command line needed, but AFAIK boost is not one of them.
How to automate this process? Well, you do not. You just include what is needed and follow the linker errors to discover further needs.
You can find which static library includes a symbol with something like:
$ nm --print-file-name --defined-only --demangle /usr/lib/*.a 2> /dev/null | \
grep -q 'boost::chrono::steady_clock::now()'
/usr/lib/libboost_chrono.a:chrono.o:0000000000000090 T boost::chrono::steady_clock::now()
But why it works in dinamic setting?
Most of my make files have the following notes (from when I found the answer back when I needed it ... sorry, I don't know where I found it.)
Note - when a build using '-l' finds the .so version of that library (so - shared object) and the same .a archive is also there, g++ prefers the .so over the .a.
However, you can still achieve static link by fully specifying the path to .a.
Example:
$(CC) $(CC_FLAGS) $< /usr/local/lib/libboost_chrono.a -o $# ...
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sometimes, archive code refers to symbols in yet another archive.
i.e. -lyyy_i686 uses some function from -lxxx_i686, and xxx was listed first in the build command. It is possible that a symbol might remain unresolved and the linker fails.
When this happens, try adding xxx a second time to the end of the archive list
from: -lxxx_i686 -lyyy_i686 -lrt -pthread
becomes -lxxx_i686 -lyyy_i686 -lrt -pthread -lxxx_i686
^^^^^^^^^_____________________________^^^^^^^^^^
The preceeding presumes only .a libraries are findable (no xxx.so)
Another alternative, you can command the linker to search an archive multiple times
-lyyy_i686 -lrt -pthread -(-lxxx_i686-)
tell gcc to link this library as many times as needed __^^__________^^
Again, only .a is findable
Finally, if you choose to link with the .so, note that this does not pull in the entire lib into the current build target. Instead, at runtime, the .so is loaded into memory, if it is not already there. This happens after the program starts. For most applications, this is considered a 'small' start-up performance hit as the program adjusts its memory map (automagically behind the scenes). I believe I once found that the app itself can control when to load the .so.
The use of .so pulls in the entire library to memory (if not already installed). Thus, there would not be any missing symbols, in my xxx vs yyy scenario above, the .so pulls all the symbols, whether used or not.
And once again, when trying to load xxx using "-lxxx_i686", the linker prefers to pull in libxxx_i686.so even when the libxxx_i686.a is in the same directory.
When compiling our project, we create several archives (static libraries), say liby.a and libz.a that each contains an object file defining a function y_function() and z_function(). Then, these archives are joined in a shared object, say libyz.so, that is one of our main distributable target.
g++ -fPIC -c -o y.o y.cpp
ar cr liby.a y.o
g++ -fPIC -c -o z.o z.cpp
ar cr libz.a z.o
g++ -shared -L. -ly -lz -o libyz.so
When using this shared object into the example program, say x.c, the link fails because of an undefined references to functions y_function() and z_function().
g++ x.o -L. -lyz -o xyz
It works however when I link the final executable directly with the archives (static libraries).
g++ x.o -L. -ly -lz -o xyz
My guess is that the object files contained in the archives are not linked into the shared library because they are not used in it. How to force inclusion?
Edit:
Inclusion can be forced using --whole-archive ld option. But if results in compilation errors:
g++ -shared '-Wl,--whole-archive' -L. -ly -lz -o libyz.so
/usr/lib/libc_nonshared.a(elf-init.oS): In function `__libc_csu_init':
(.text+0x1d): undefined reference to `__init_array_end'
/usr/bin/ld: /usr/lib/libc_nonshared.a(elf-init.oS): relocation R_X86_64_PC32 against undefined hidden symbol `__init_array_end' can not be used when making a shared object
/usr/bin/ld: final link failed: Bad value
Any idea where this comes from?
You could try (ld(2)):
--whole-archive
For each archive mentioned on the command line after the --whole-archive option, include every object file in the
archive in the link, rather than searching the archive for the required object files. This is normally used to turn
an archive file into a shared library, forcing every object to be included in the resulting shared library. This
option may be used more than once.
(gcc -Wl,--whole-archive)
Plus, you should put -Wl,--no-whole-archive at the end of the library list. (as said by Dmitry Yudakov in the comment below)
I'm writing a small C program that uses librt. I'm quite surprised that the program won't compile if I place the link flag at the start instead of at the end:
At the moment, to compile the program I do:
gcc -o prog prog.c -lrt -std=gnu99
If I were to do the following, it will fail to find the functions in librt:
gcc -std=gnu99 -lrt -o prog prog.c
Yet, this works with other libraries. I found the issue when attempting to use a simple Makefile. make actually compiled prog.c without liking first (using -c flag) and then did the linking.
This is the Makefile:
CC = gcc
CFLAGS = -std=gnu99
LIBS= -lrt
LDFLAGS := -lrt
prog: prog.o
$(CC) -o prog prog.c -lrt -std=gnu99
The output I would get when typing make would be:
gcc -std=gnu99 -c -o prog.o prog.c
gcc -lrt prog.o -o prog
prog.o: In function `main':
prog.c:(.text+0xe6): undefined reference to `clock_gettime'
prog.c:(.text+0x2fc): undefined reference to `clock_gettime'
collect2: ld returned 1 exit status
make: *** [buff] Error 1
I have now crafted a Makefile that puts the linking at the end of the gcc line, however I'm puzzled why it doesn't work if the linking flag is at the start.
I would appreciate if anybody can explain this to me. Thanks.
As the linker processes each module (be it a library or a object file), it attempts to resolve each undefined symbol while potentially adding to its list of undefined symbols. When it gets to the end of the list of modules, it either has resolved all undefined symbols and is successful or it reports undefined symbols.
In your case, when it processed librt, it had no undefined symbols. Processing proc resulted in clock_gettime being an undefined symbol. gcc will not go back and look in librt for the undefined symbols.
For that reason, you should always have your code first, followed by your libraries, followed by platform provided libraries.
Hope this helps.
From the ld (the GNU linker) documentation (http://sourceware.org/binutils/docs/ld/Options.html#Options):
The linker will search an archive only once, at the location where it is specified on the command line. If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an object appearing later on the command line will not cause the linker to search the archive again.
So if you specify the library too early, the linker will scan it, but not find anything of interest. Then the linker moves on to the object file produced by the compiler and finds references that need to be resolved, but it has already scanned the library and won't bother looking there again.
the title of this question is an exact dupe, but the answers in that question don't help me.
I have a bunch of object files packed in a static library:
% g++ -std=c++98 -fpic -g -O1 -c -o foo.o foo.cpp
% g++ -std=c++98 -fpic -g -O1 -c -o bar.o bar.cpp
% ar -rc libsome.a foo.o bar.o
I'd like to generate libsome.so from libsome.a instead of the object files, but the library is really barebones:
% g++ -std=c++98 -fpic -g -O1 -shared -o libsome.so libsome.a
% nm -DC libsome.so
0000xxxx A _DYNAMIC
0000xxxx A _GLOBAL_OFFSET_TABLE_
w _Jv_RegisterClasses
0000xxxx A __bss_start
w __cxa_finalize
0000xxxx A _edata
0000xxxx A _end
0000xxxx T _fini
0000xxxx T _init
the static library is ok, or at least I'm perfectly able to link it to an executable and have it run the contained functionality. also, everything is fine if I create libsome.so from foo.o and bar.o.
Assuming you're using the GNU linker, you need to specify the --whole-archive option so that you'll get all the contents of the static archive. Since that's an linker option, you'll need -Wl to tell gcc to pass it through to the linker:
g++ -std=c++98 -fpic -g -O1 -shared -o libsome.so -Wl,--whole-archive libsome.a
If you were doing something more complicated where you want all of library some but only the part of library support needed by libsome, you would want to turn off whole archive after you've used it on libsome:
... -Wl,--whole-archive libsome.a -Wl,--no-whole-archive libsupport.a
If you're not using the GNU linker, you'll need to see if your linker supports it and what it's called. On the Sun linker, it's called -z allextract and -z defaultextract.
Your files are not being pulled in from the .a file because nothing is referencing them. Why are you making a .a file first? You'd have better luck making a .o file by linking them with the -r option.